Persistence & recovery

External persistence is an optional background archive that uploads local session checkpoints and event logs to a secondary filesystem directory, AWS S3, Cloudflare R2, or Google Cloud Storage (persistence in brain.json).

Brain always keeps active session state on local disk first (metadata.sqlite for session records, plus append-only events.jsonl and native/ files in data_dir, .brain/ by default) so turns never block on remote storage.

Note: Because uploads run asynchronously in the background, an ungraceful machine crash can lose events written after the last uploaded checkpoint.

Before you beginCopy Anchor Link

• A working Brain configuration file (brain.json).
• Write access to a local archive directory or a bucket in AWS S3, Cloudflare R2, or Google Cloud Storage.
• Optional: a PostgreSQL connection string (postgres_env) if you need cross-instance cold-session discovery (GET /sessions).

Configure a storage providerCopy Anchor Link

Add a persistence block to your host configuration file. Using the filesystem provider is the fastest way to test checkpointing and recovery on a single machine:

{
  "provider": "filesystem",
  "directory": ".brain-archive",
  "interval_seconds": 5,
  "queue": 128
}

Object storage providersCopy Anchor Link

• Least-privilege permissions: The Brain server only needs object read/write (GET and PUT, or object read/create/update on GCS) under its configured prefix; no bucket administration permissions are required.
• Bucket isolation: Keep internal checkpoint storage separate from any S3/R2 buckets exposed to agents as tools.
• No credential archiving: Brain never archives provider API keys or host credential files. When a session is restored on a new server, model and tool credentials come from the new host's configuration.

Note: Before you rely on a store, verify credentials, IAM permissions, and bucket access in your own account, and run a restore drill.

Optional PostgreSQL discovery index (postgres_env)Copy Anchor Link

Set postgres_env to the name of an environment variable holding a PostgreSQL connection string (postgres://...) to maintain an eventually consistent session index across instances:

• Hosted PostgreSQL (Neon, Supabase, RDS): Put the provider's connection string in the environment variable named by postgres_env with TLS enabled. Use a direct connection or session pool initially, because transaction-pool prepared-statement behavior varies by provider. Brain bounds each process to at most 4 open database connections.
• Discovery only: PostgreSQL stores an eventually consistent session index for GET /sessions and cold-session lookup; it does not lock or coordinate session ownership.

How checkpoints workCopy Anchor Link

Brain uploads events.jsonl and append-only native/ harness files in immutable 64 KiB blocks alongside bounded tail segments. It writes the checkpoint manifest (sessions/<id>/checkpoints/<checkpoint_id>) only after every referenced block is in place, then updates the mutable sessions/<id>/latest pointer (using generation preconditions on immutable objects and replaceable writes on latest) and optional PostgreSQL index. Already uploaded 64 KiB blocks are reused on subsequent turns instead of re-uploading the full conversation history.

On startup, Brain scans local disk and queues any unsynced sessions. Transient storage failures retry up to 5 times with exponential backoff, while permanent configuration or credential errors stop immediately. If the sync queue (queue, default 128) fills up, Brain rejects new writes with 429 overloaded and marks GET /health/ready as 503 not_ready until the backlog drains.

Flush and inspect sync statusCopy Anchor Link

• GET /sessions/{id}/sync returns the latest published checkpoint status, pending upload state, and any sync error.
• POST /sessions/{id}/flush (requires manage: true) forces an immediate upload of local event history and the latest completed native checkpoint and returns a SyncStatus object:

Note: If you call POST /sessions/{id}/flush while a turn is still running, history_bytes may include events written beyond resume_offset. Flushing does not pause or trigger model execution.

Move a session to another serverCopy Anchor Link

Brain does not run distributed consensus or merge writes from two servers running the same session. Your platform must stop the old owner before starting the new one:

1. Quiesce the old instanceCopy Anchor Link

Stop routing new input to the old instance and let the current turn finish (or send user.interrupt via POST /sessions/{id}/events and wait for the session to settle).

2. Flush and record the checkpoint IDCopy Anchor Link

Call POST /sessions/{id}/flush on the old instance and save the returned checkpoint reference from the response.

3. Re-route to the new instanceCopy Anchor Link

Stop the old owner and route the session to the new instance. The new instance must have the same runtime version, harness configuration, tool bindings, and external storage access.

4. Restore the explicit checkpointCopy Anchor Link

Call POST /sessions/{id}/restore on the new instance with {"checkpoint": "RETURNED_REFERENCE"}.

5. Resume the sessionCopy Anchor Link

Call POST /sessions/{id}/resume to move the restored session from interrupted to idle, then send new input.

When a server receives GET /sessions/{id} or GET /sessions/{id}/events for a session that is not on local disk, it automatically loads the latest published checkpoint from external storage. During a planned handoff, always call POST /sessions/{id}/restore with the explicit checkpoint ID from step 2 so a stale local cache or delayed upload cannot pick an older boundary.

Note:

• When restore replaces an existing local session directory, Brain renames the old directory to SESSION_ID-replaced-ID so operators can inspect any unsynced local tail before deleting it.
• Restoring a session recovers its transcript and native model context; it does not roll back or restart external tool jobs, sandboxes, or browser sessions.
• Configure bucket object lifecycle policies so they never expire 64 KiB blocks referenced by retained checkpoints.

Next stepsCopy Anchor Link

• Host configuration reference
• Persistence adapters
• Operations & observability
• HTTP API reference