Back to blog

The Brain Reacts Now

Three weeks ago I shipped the second milestone of The Brain — the scheduler daemon, cron triggers, workflows that read their previous run, an opt-in HTTP endpoint. That was M2. The Brain could run unattended on a clock.

Today M3 is done. The Brain now reacts — to HTTP requests, and to filesystem changes.

Why this matters

M2 made The Brain worth running unattended on a schedule. M3 makes it react to things that happen. The hardest, most useful workflow automations are the reactive ones — the workflow that fires when a customer signs up, the workflow that processes a file the moment it lands on disk, the workflow that wakes up because another system has news.

M1 was the runner. M2 made the runner unattended. M3 makes the runner reactive. M1 + M2 + M3 together is the trigger surface most people actually need.

What M3 ships

Webhook triggers. Register any workflow as a webhook endpoint. The Brain prints a secret once, you save it, and from that moment on, any HTTP caller with the secret can fire the workflow over the network.

docker compose exec brain brain register-webhook examples/webhook_handler.py

The CLI prints the HMAC secret exactly once — same caller-side-storage discipline as a GitHub personal access token. There's no brain show-webhook-secret command by design; if you lose the secret, you unregister and re-register to issue a fresh one.

Fire it from anywhere that can compute an HMAC signature:

SECRET=<your-saved-secret>
BODY='{"hello":"world"}'
SIG="sha256=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')"

curl -X POST http://localhost:8001/webhook/webhook-handler \
    -H "X-Brain-Signature: $SIG" \
    -H "Content-Type: application/json" \
    -d "$BODY"

The X-Brain-Signature: sha256=<hex> header convention is identical to GitHub's X-Hub-Signature-256 — so existing webhook senders work without translation. The endpoint runs the workflow synchronously and returns the run metadata. Wrong signature is 401. Unknown workflow name is 404, same shape as a disabled webhook, so existence is not leaked through the response code.

File watcher triggers. Register a workflow to fire when something changes on disk. The Brain runs a separate watcher daemon that observes the directory and fires the workflow on filesystem events.

docker compose exec brain-watcher brain register-watcher examples/markdown_watcher.py \
    --path /data/watched --events modified

--events accepts any combination of created, modified, deleted. The watcher daemon picks up the new registration on its next 10-second sync. A 500ms debounce per (workflow, path) coalesces multiple filesystem events from a single editor save into one workflow run — the most common case where a save emits both modified and created events within a few milliseconds.

The watcher runs in its own container behind the watcher compose profile. If the watcher crashes, the scheduler from M2 keeps running. If the scheduler crashes, the watcher keeps watching. Isolation by container, not by retry loops.

A trigger placeholder family. Workflows triggered by a webhook or file event can read the inbound payload via a new placeholder family:

ShellStep(
    name="received",
    command="echo got event={trigger.event} body={trigger.body}",
)

Four placeholders are available wherever string substitution works:

Referencing {trigger.X} on a workflow you ran manually fails the step with a clear error — same strict-failure shape as M2's {previous.X} placeholder when no previous run exists.

The four classical trigger types — manual, cron, webhook, file — are now all there. Same workflow model. Same persistence model. Same brain history and brain show view of every run. The only thing that varies across triggers is which mechanism started the run, and what shows up in trigger_context.

How it works under the hood

The webhook endpoint reads the raw request body before any JSON parsing — the HMAC signature is computed over the bytes the sender sent, not over a re-encoded representation, so signature verification is byte-exact. The verifier is hmac.compare_digest for constant-time comparison, and every failure path — wrong signature, missing signature header, malformed hex, unknown algorithm — runs through the same comparison against a placeholder digest. There is no early len(a) != len(b) branch. An end-to-end timing-attack regression test measures wall-clock variance between "wrong-length" and "right-length-wrong-value" failures across 2000 iterations and asserts the ratio stays under 10x. If a future refactor introduces a length-check shortcut, the test breaks loudly.

The watcher daemon uses the watchdog library — the same cross-platform filesystem-events abstraction that hot-reload frameworks and IDE file-syncing tools use. One watchdog.Observer per enabled watcher row, each observing exactly one directory, no recursion. The watcher daemon and the scheduler daemon both write heartbeats into the same daemon_heartbeats table, but their daemon IDs are different: the scheduler uses the container hostname, the watcher appends :watcher. The two daemons run in separate containers and never collide on crash recovery — the watcher's recovery sweep is scoped to trigger_context->>'event' = 'file', the scheduler's is broader, and the two queries are mutually disjoint.

The {trigger.X} resolver is a single regex pass through the runner's existing placeholder substitution. Resolution order is {previous.X} first (M2), then {trigger.X} (M3), then {step_name} (M1). The three families are independent — a single step can freely mix them, e.g. echo prior={previous.summary} live={trigger.event} now={current_step}.

Three locked invariants worth naming because they shape the design space.

Disabled webhooks return 404, not 401. Same shape as nonexistent. The webhook name is not a secret in this threat model; the bearer token at the boundary is. If you can list webhooks, you already have privileged access. Returning 401 for disabled-but-existing rows would leak the existence of the registration to anyone who tries the path — pointless given the threat model, harmful in the abstract. Pinned as a regression test: any future refactor that adds a constant-time-equal-lookup must break the test and surface as an architectural decision.

500ms debounce per (workflow, path). In-memory only, lost on daemon restart, which is fine because crash recovery re-fires from current filesystem state and any in-flight transient state is by definition stale. The boundary is exact — 499ms after a fire blocks the next event, 501ms doesn't — and pinned by tests at the unit-function level and the module-constant level.

Sequential within process, concurrent across processes. Two simultaneous webhook calls to the same workflow queue inside the API process. Two simultaneous file events to the same watcher queue inside the watcher process. But the API, scheduler, and watcher daemons all run workflows in parallel because they're separate processes against the same database. The Brain doesn't have a global work queue — workflows are independent at the process boundary, and the database absorbs concurrent INSERTs.

What M3 does not do

Worth naming so the next post isn't an apology.

What's next

Milestone 4 adds MCP tool calling as a step type, plus a pluggable LLM provider abstraction. That's the milestone where The Brain becomes ecosystem-aware — any MCP server in your environment becomes a callable step, not just Memory Vault.

The full roadmap and milestone progress table live in the repo's README.

Try it

git clone https://github.com/MihaiBuilds/the-brain
cd the-brain

# bring up everything: scheduler + webhook API + file watcher
THE_BRAIN_API_TOKEN=any-value docker compose --profile api --profile watcher up -d

# register a webhook (saves the secret to stdout — copy it now)
docker compose exec brain brain register-webhook examples/webhook_handler.py

# register a file watcher (must run from inside the watcher container)
docker compose exec brain-watcher brain register-watcher examples/markdown_watcher.py \
    --path /data/watched --events modified

# see all your triggers in one place
docker compose exec brain brain list-triggers

Drop a file in ./watched, sign and POST to http://localhost:8001/webhook/webhook-handler, and the runs land in brain history alongside any manual or scheduled runs from M1 and M2.

The repo has the longer version with the full HMAC signing recipe, the trigger-placeholder reference, and the lifecycle commands for both trigger types.

Follow the build

Watch the repo to follow along as each milestone ships.