Brain Server — API Contract (/recall + /ingest)
Wire contract for the brain-server HTTP API. The JSON shapes here are the source of truth; the Rust
serdestructs are kept equal to these shapes.Status:
/recalland/ingestare both implemented and live in the current source (seesrc/handlers/recall.rs,src/handlers/ingest.rs). They supersede the legacy/searchand/ingest/markdown; the legacy endpoints remain for direct/CLI compatibility (documented inREADME.mdandSPECS.md, out of scope here).Versioning: the server reports
SERVER_VERSION = env!("CARGO_PKG_VERSION")via/versionand/health, and sets anX-Api-Version: <semver>response header on every route. Contract version:api v1.
Versioning & deprecation policy
Applies from v0.9.5 (“Inspect” M3) onward, before third parties depend on the API surface.
- Version discovery. Every response carries
X-Api-Version: <semver>(the crate version fromCargo.toml). Clients SHOULD log/record it; a major bump (1.x→2.x) signals a breaking wire change. - Structured queries. The canonical query contract is the
QueryDoc(seesrc/search/query.rsandopenapi.yaml#/components/schemas/QueryDoc), sent toPOST /recall. The legacyGET /search(flatq/lex/source) andPOST /addremain functional but are deprecated. - Deprecation signal. Deprecated routes return an RFC 8594
Deprecationheader (e.g.Deprecation: version="0.9.5"). The header names the version in which the route entered deprecation, not the version it will be removed. Removal only happens on a major-version boundary, and only after a minimum of one minor release of overlap with the replacement route. - Migration mapping.
Deprecated Replacement GET /search?q=...POST /recallwithQueryDoc(structuredlex,sources,intent,explain)POST /addPOST /ingest/memory(raw body) orPOST /ingest/markdown(with title) - Stability promise. Within a major version, existing response shapes are additive (new optional fields only). A removed field or changed type is a breaking change and requires a major bump.
The full machine-readable route set lives in
openapi.yaml(served atGET /openapi.yaml); keep the two in sync — thetest_openapi_covers_routesunit test enforces it.
0. Conventions
| Concern | Rule |
|---|---|
| Content-Type | application/json (UTF-8) for all request/response bodies with a body |
| Auth | Authorization: Bearer <token> when a server-side token is configured (AUTH_TOKEN, or AUTH_TOKEN_FILE pointing at a 0600 file — the latter is preferred). Loopback may be exempt — server policy. Constant-time compare. |
| Unknown fields | Ignored on deserialize (forward-compatible). Servers MUST NOT reject unknown keys. |
| Missing optional fields | Omitted, not null. With exactOptionalPropertyTypes on the TS side, undefined keys are not serialized (conditional spread). |
| IDs | Knowledge IDs are i64 (serialized as JSON number). Entity/relation IDs are not exposed over the wire by these endpoints. |
| Strings | UTF-8; all bounds are UTF-8 byte lengths unless noted. |
| Errors | Uniform envelope (§5). Never leak internals (paths, SQL, stack). |
| Timeouts | Server enforces a 30 s per-request deadline + an 8 s /recall search budget; client also sets AbortController. |
Field bounds (enforced server-side → 400 on violation)
| Field | Bound | Error code |
|---|---|---|
query | 1 ≤ len ≤ 2,000 (utf8 bytes) | query_empty / query_too_long |
limit | 1 ≤ n ≤ 100 | limit_out_of_range |
title | 1 ≤ len ≤ 500 | title_invalid |
content | 1 ≤ len ≤ 1,000,000 (1 MiB) | content_empty / content_too_large |
domain | matches ^[a-z0-9][a-z0-9_-]{0,62}$ | domain_invalid |
entity/relation name | 1 ≤ len ≤ 100, ^[A-Za-z0-9 _-]+$ | name_invalid |
entity type | len ≤ 64 | entity_invalid |
relation type | 1 ≤ len ≤ 64, ^[a-z0-9_]+$ (snake_case) | relation_invalid |
arrays (entities/relations) | ≤ 200 each per request | too_many_entities / too_many_relations |
Domain names are lowercase by convention. The server normalizes to lowercase (trim + lower) before comparison (so
Health→health). Entity/relation names are also normalized to lowercase internally; their surrounding whitespace is collapsed. A well-formed but unregistered forceddomainresolves todomain_invalidtoday (thedomain_unknowndistinction is reserved for a future per-domain registry; see §2).
1. Common types
Domain
A domain name string (see bounds above). The reserved domain global is the fallback sink.
Entity
{ "name": "vitamin d3", "type": "supplement" }
name— required, the entity surface form (case-insensitive unique within a domain).type— optional free-form label (e.g."supplement","person","concept").
Relation
{ "from": "vitamin d3", "to": "inflammation", "type": "helps" }
from/to— entity names (must match anEntity.namein the same payload OR an existing entity in the domain; server upserts entities as needed).type— snake_case relation label.
RecallHit
{
"id": 42,
"title": "Vitamin D3 notes",
"content": "Vitamin D3 supports immune function...",
"score": 0.87,
"domain": "health",
"source": "both",
"provenance": { "vector_rank": 0, "fts_rank": 1, "fused_score": 0.0327 }
}
| Field | Type | Always? | Notes |
|---|---|---|---|
id | integer | yes | knowledge id |
title | string | null | no | omitted if absent |
content | string | yes | the matched chunk with a bounded, faithful snippet window |
score | number (float) | yes | normalized similarity/fusion score |
domain | string | no | the domain the hit came from (present when provenance=true) |
source | "vector" | "fts" | "both" | "graph" | no | retrieval path (present when provenance=true) |
provenance | object | no | per-retriever ranks + fused score (present when provenance=true) |
provenance (per-hit)
The shape of RecallHit.provenance (defined in src/search/mod.rs):
| Field | Type | Notes |
|---|---|---|
vector_rank | integer | omitted | rank the vector retriever assigned (0 = best) |
fts_rank | integer | omitted | rank the FTS5 retriever assigned |
fused_score | number | omitted | RRF-fused score |
rerank_score | number | omitted | cross-encoder score (only if the rerank tier ran) |
rerank_truncated | boolean | doc was length-capped before reranking |
prf_expanded | boolean | hit surfaced via the PRF-expanded pass |
top_retrieval_mode | "vector" | "fts" | "both" | omitted | which retriever(s) contributed the top result |
retrieval_strategy | string | omitted | overall strategy, e.g. hybrid or hybrid_prf |
quality_assessment | object | omitted | heuristic confidence + recommendation (see src/search/quality.rs) |
prf_decision | object | omitted | why PRF did/didn’t fire |
2. POST /recall — deterministic recall
The server does everything: embed the query → auto-route via domain centroids → search (hybrid vec0 + FTS5, RRF fusion) → optional PRF query expansion → optional cross-encoder rerank → cross-domain fallback on miss → cap → return.
Request
{
"query": "supplements for inflammation",
"limit": 3,
"domain": "health", // optional: force a domain (disables auto-routing)
"strict": false, // optional: true = no cross-domain fallback
"provenance": true, // optional: include per-hit domain + source + provenance + telemetry
// ── optional structured-query overrides (power tools) ──
"source": "structured", // filter: ingest kind, retrieval leg, or both (see table)
"since": "2026-01-01", // ISO-8601 / RFC3339; rows with created_at > since
"lex": "inflammation -fever", // lexical (FTS5) query override
"vec": "immune support", // semantic embedding-query override
"hyde": "Vitamin D3 reduces...", // hypothetical-answer embedding override (beats `vec`)
"intent": "lookup" // free-form intent label, recorded for provenance
}
| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
query | string | yes | — | the user turn / search text |
limit | integer | no | 5 | capped 1–100 |
domain | string | no | (auto-route) | force a specific domain |
strict | boolean | no | false | disable fallback fan-out |
provenance | boolean | no | false | include domain/source/provenance per hit + telemetry (domainsSearched is always present) |
source | string | no | — | v1.13.3: an ingest kind (memory·markdown·structured·manual·vault) filters in SQL; a retrieval leg (vector·fts·graph) filters post-fusion; both is unrestricted. Unknown values return 422. |
sources | string[] | no | — | OR filter over ingest kind (memory·markdown·structured·manual·vault) — filters the source column, NOT source URIs. |
since | string | no | — | ISO-8601 (RFC3339 or YYYY-MM-DD HH:MM:SS). Validated inside the search path; a malformed value is silently swallowed on the recall path today (the failing target contributes no hits) rather than surfacing a 400 |
lex | string | no | — | lexical (FTS5) query override (exact terms, phrases, -exclusions) |
vec | string | no | — | semantic embedding-query override |
hyde | string | no | — | hypothetical-answer embedding override; takes priority over vec |
intent | string | no | — | free-form intent label, recorded for provenance |
Response — 200 OK
{
"hits": [
{ "id": 42, "title": "Vitamin D3 notes", "content": "...", "score": 0.87, "domain": "health", "source": "both", "provenance": { "..." : "..." } },
{ "id": 88, "title": "Omega-3", "content": "...", "score": 0.71, "domain": "global", "source": "fts" }
],
"domain": "health",
"domainsSearched": ["health", "global"],
"telemetry": { "embed_ms": 1.2, "vector_ms": 3.4, "fts_ms": 1.1, "fusion_ms": 0.1, "confidence": 0.78 }
}
| Field | Type | Always? | Notes |
|---|---|---|---|
hits | RecallHit[] | yes | ordered by descending score; length ≤ limit |
domain | string | yes | the primary domain chosen by routing (or the forced domain) |
domainsSearched | string[] | yes | domains of the returned hits (empty array when no hits). Always present (v1.13.3); no longer gated on provenance. |
telemetry | object | no | per-stage retrieval telemetry. Present when provenance=true. |
telemetry (per-response)
The shape of RecallResponse.telemetry (defined in src/search/mod.rs::SearchTelemetry):
| Field | Type | Notes |
|---|---|---|
embed_ms / vector_ms / fts_ms / fusion_ms / prf_ms / rerank_ms | number | per-stage latency (ms) |
retrieval_ms_vec / retrieval_ms_fts | number | retrieval latency excluding embedding |
vec_candidates / fts_candidates / fused_count | integer | candidate counts before/after RRF |
rrf_k | integer | RRF k parameter (60) |
confidence | number | heuristic quality-estimator score (0–1) |
recommendation | string | omitted | "return" / "run_prf" / "run_reranker" / "increase_top_k" / "clarify_query" |
intent / embedding_query | string | omitted | effective intent / embedding query used |
Routing semantics
domainprovided → search only that domain. Unknown/unresolvable →400 domain_invalid.domainomitted (auto-route): a. Embed query once (model2vec). b. Compare to every domain centroid (int8/binary, Hamming/cosine). Rank domains. c. Primary domain = top centroid aboveDOMAIN_CONFIDENCE_THRESHOLD(0.55). d. If none above threshold → primary =global.- Search the primary domain (hybrid vec0 KNN + FTS5 BM25, RRF fusion; optional PRF + rerank).
- Fallback (unless
strict=true): if no confident route → fan out across all known domains +global; merge by score; tag each hit’sdomain. - Cap to
limit; return.
Empty result is not an error —
200withhits: [].
Errors
| Status | Code | When |
|---|---|---|
| 400 | query_empty / query_too_long | missing/oversized query |
| 400 | query_rejected | query matches a blocked prompt-injection pattern |
| 400 | limit_out_of_range | limit outside 1–100 |
| 400 | domain_invalid | malformed or unresolvable forced domain |
| 401 | unauthorized | missing/invalid bearer |
| 429 | rate_limited | per-IP/domain rate limit breach |
| 503 | recall_unavailable | search task failed or exceeded the 8 s budget |
domain_unknownis reserved for a future per-domain registry that distinguishes “well-formed but unregistered” from “malformed.” Today both resolve todomain_invalid.
3. POST /ingest — structured store (the KG write path)
Stores a knowledge entry + its embedding (auto-resolved domain if omitted), plus optional explicit entities/relations that populate the domain’s knowledge graph. The server trusts the caller’s graph data after validation (no server-side extraction — the annotation engine was retired in v0.9.0).
Request
{
"title": "Vitamin D3 benefits",
"content": "Vitamin D3 supports immune function and helps with inflammation...",
"domain": "health", // optional: resolved domain if omitted
"entities": [
{ "name": "vitamin d3", "type": "supplement" },
{ "name": "inflammation", "type": "condition" }
],
"relations": [
{ "from": "vitamin d3", "to": "inflammation", "type": "helps" }
]
}
| Field | Type | Required | Notes |
|---|---|---|---|
title | string | yes | 1–500 chars (trimmed) |
content | string | yes | 1–1,000,000 chars (not trimmed) |
domain | string | no | force domain; omit → resolved to global |
entities | Entity[] | no | upsert into the domain KG |
relations | Relation[] | no | upsert; from/to upserted as entities if new |
Response — 200 OK
{
"id": 42,
"status": "created",
"domain": "health",
"entitiesAdded": 2,
"relationsAdded": 1
}
| Field | Type | Always? | Notes |
|---|---|---|---|
id | integer | yes | knowledge id. On duplicate, returns the existing knowledge id. |
status | "created" | "duplicate" | yes | duplicate = content_hash already present (xxh3-64 of content) |
domain | string | yes | the domain actually written to (forced or global) |
entitiesAdded | integer | yes | count of entities in the request that were processed (upsert is idempotent, so this is the request count, not the delta of newly-inserted rows) |
relationsAdded | integer | yes | count of relations in the request that were processed (same caveat) |
Behavior
- Dedup: content hashed (xxh3-64); exact dup →
status: "duplicate", the existing id, no embedding work, no entity/relation mutation (entitiesAdded: 0,relationsAdded: 0). - Domain resolution: if
domainomitted → resolved toglobal(no centroid routing on the write path today). After a successful write the server best-effort recomputes that domain’s centroid so future/recallauto-routing can target it. - Entities/relations are scoped to the resolved domain.
INSERT OR IGNOREsemantics (idempotent).from/toinrelations[]are resolved to existing entity rows (they must already exist inentities[]or in the domain — relation insert fails if a referenced entity cannot be resolved). - Embedding: content is embedded once (model2vec) and stored in
vec_knowledgeas int8 + binary quantized vectors. The legacy f32 JSONembeddingscolumn is no longer written. - Atomicity: knowledge + vec0 + entities + relations in one SQLite transaction.
Errors
| Status | Code | When |
|---|---|---|
| 400 | title_invalid / content_empty / content_too_large | bounds violations |
| 400 | name_invalid | bad entity/relation name (empty, > 100, bad charset) |
| 400 | entity_invalid | entity type > 64 chars |
| 400 | relation_invalid | bad relation type (empty, > 64, not snake_case) |
| 400 | too_many_entities / too_many_relations | array > 200 |
| 400 | domain_invalid | malformed or unresolvable forced domain |
| 401 | unauthorized | auth |
| 413 | (bare status) | body > 1 MiB (MAX_REQUEST_SIZE), enforced by the HTTP RequestBodyLimitLayer before the handler runs — returned as a plain 413, not the JSON envelope. (HandlerError::payload_too_large exists but is not invoked by this route.) |
| 429 | rate_limited | per-IP/domain write limit |
| 500 | internal_error | DB/embedding/transaction failure |
4. Supporting endpoints
GET /health → 200
Illustrative example — the
versionisenv!("CARGO_PKG_VERSION")at runtime andcapacityis present only when the connection pool is not momentarily exhausted.
{
"status": "ok",
"version": "1.27.22",
"model": "minishlab/potion-retrieval-32M",
"system": { "memory_used_mb": 220, "memory_total_mb": 4096, "memory_percent": 5.4 },
"pool": { "connections": 2, "idle_connections": 1, "busy_connections": 1 },
"backup": { "ok": true },
"webhook": { "replay_secs": 600, "timestamp_required": 0, "scheme": "legacy" },
"otel": { "enabled": false, "endpoint": "http://127.0.0.1:4317" },
"integrity": { "chain_ok": true, "last_checked_at": "...", "chain_head": "..." },
"capacity": { "status": "ok", "docs": 430, "db_mib": 12, "rss_mib": 84 }
}
The primary consumer probes this to confirm the server is up (it only
reads status). On failure the server returns { "status": "error", "version": "...", "error": "..." }.
version is env!("CARGO_PKG_VERSION").
DELETE /memory/{id} → 200 / 404
{ "deleted": true }
Cascades to the entry’s vec_knowledge row (cleaned explicitly — vec0 has no FK),
embeddings (FK CASCADE), and owned relations (FK SET NULL); the FTS trigger removes the
FTS row. A tombstones row records the deletion for provenance. id is parsed as i64
(non-numeric → 400). 404 body: { "error": { "code": "not_found", "message": "..." } }.
GET /domains → 200 (ops/debug)
{
"domains": [
{ "name": "global", "entries": 1307, "entities": 2341, "relations": 1892, "multi_db": false },
{ "name": "health", "entries": 412, "entities": 2341, "relations": 1892, "multi_db": false }
]
}
Not used by the recall hot path, but useful for the brain CLI and for surfacing
knownDomains.
v1.0.0 lifecycle routes (per the plan M5):
POST /domainsbody{"name": "health"}— create or warm a domain. Idempotent; returns201on first open,200if already present.DELETE /domains/{name}?confirm={name}— drop ALL data for the domain and VACUUM. Theglobaldomain is protected. The?confirm=<exact-name>query param is REQUIRED so a typoed URL or replay can’t destroy data by accident.POST /domains/{name}/vacuum— reclaim free pages. Cheap, safe under load.GET /domains/{name}/export— stream a consistent snapshot viaVACUUM INTO. Returnsapplication/octet-stream+Content-Disposition: attachment.POST /domains/{name}/import— restore a snapshot into a NEW domain. Body is the raw bytes from a prior export. Target must not already exist;globalis protected. Atomic temp-file + rename; migration runs on the imported pool.
Per-domain counts. In shim mode (
BRAIN_MULTI_DB=false, the default) the registry enumerates thedomaincolumn on the shared pool — entities and relations are global totals in that mode. In multi-db mode each domain has its own file and the counts are genuinely domain-scoped.
5. Error envelope (uniform)
Every non-2xx response uses this shape:
{
"error": {
"code": "domain_invalid",
"message": "domain 'heath' is not registered",
"details": { "max": 200 }
}
}
| Field | Type | Always? | Notes |
|---|---|---|---|
error.code | string | yes | machine-readable snake_case code (see per-endpoint tables) |
error.message | string | yes | safe human text; never includes paths/SQL/secrets |
error.details | object | no | structured context (e.g. {min, max} for range errors) |
Consumers SHOULD treat any non-2xx as an error, distinguishing 404 from
other statuses. 401 unauthorized MUST be surfaced (not silently swallowed) for
security visibility.
6. Rust (Axum + serde) — canonical definitions
The shared response/error types live in src/handlers/mod.rs; the per-endpoint request
types live alongside their handlers. Uses crates already in Cargo.toml (serde,
serde_json, axum 0.8).
src/handlers/mod.rs — shared types
#![allow(unused)]
fn main() {
use axum::http::StatusCode;
use axum::response::IntoResponse;
use serde::{Serialize};
use serde_json::Value;
#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum HitSource { Vector, Fts, Both, Graph }
#[derive(Debug, Serialize)]
pub struct RecallHit {
pub id: i64,
#[serde(skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
pub content: String,
pub score: f32,
#[serde(skip_serializing_if = "Option::is_none")]
pub domain: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub source: Option<HitSource>,
/// Per-retriever ranks + fused score. Present only when `provenance=true`.
#[serde(skip_serializing_if = "Option::is_none")]
pub provenance: Option<crate::search::Provenance>,
}
#[derive(Debug, Serialize)]
pub struct RecallResponse {
pub hits: Vec<RecallHit>,
#[serde(skip_serializing_if = "Option::is_none")]
pub domain: Option<String>,
/// v1.13.3 "SourceFix": always present (empty when no hits).
pub domains_searched: Vec<String>,
/// Per-stage retrieval telemetry. Present only when `provenance=true`.
#[serde(skip_serializing_if = "Option::is_none")]
pub telemetry: Option<crate::search::SearchTelemetry>,
}
#[derive(Debug, Serialize)]
pub struct IngestResponse {
pub id: i64,
pub status: &'static str, // "created" | "duplicate"
#[serde(skip_serializing_if = "Option::is_none")]
pub domain: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub entities_added: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub relations_added: Option<u32>,
}
#[derive(Debug, Serialize)]
pub struct ForgetResponse { pub deleted: bool }
// ---------- uniform error envelope ----------
#[derive(Debug, Serialize)]
pub struct ErrorBody { pub error: ApiError }
#[derive(Debug, Serialize)]
pub struct ApiError {
pub code: &'static str,
pub message: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub details: Option<Value>,
}
/// Handler error type → renders the uniform `ErrorBody` envelope.
#[derive(Debug)]
pub struct HandlerError { pub status: StatusCode, pub inner: ApiError }
impl IntoResponse for HandlerError {
fn into_response(self) -> axum::response::Response {
(self.status, axum::response::Json(ErrorBody { error: self.inner })).into_response()
}
}
}
src/handlers/recall.rs — request
#![allow(unused)]
fn main() {
#[derive(Debug, Deserialize)]
pub struct RecallRequest {
pub query: String,
#[serde(default = "default_limit")]
pub limit: u32,
pub domain: Option<String>,
#[serde(default)] pub strict: bool,
#[serde(default)] pub provenance: bool, // alias "explain"
#[serde(default)] pub source: Option<String>,
#[serde(default)] pub since: Option<String>,
#[serde(default)] pub lex: Option<String>, // bare string or LexSpec object
#[serde(default)] pub vec: Option<String>,
#[serde(default)] pub hyde: Option<String>,
#[serde(default)] pub intent: Option<String>,
#[serde(default)] pub sources: Vec<String>, // OR filter over ingest kind
#[serde(default)] pub profile: Option<String>,
#[serde(default)] pub include_flagged: bool,
#[serde(default)] pub as_of: Option<String>,
#[serde(default)] pub evidence: bool,
#[serde(default)] pub at: Option<String>,
#[serde(default)] pub max_context_tokens: Option<usize>,
#[serde(default)] pub gold_answer: Option<String>,
#[serde(default)] pub graph: bool,
#[serde(default)] pub include_decayed: bool,
#[serde(default)] pub memory_kind: Option<String>,
#[serde(default)] pub min_relevance: Option<String>,
#[serde(default)] pub trace: bool,
}
}
Canonical field list as of v1.20.29; the current source is authoritative — see
src/handlers/recall.rs.
### `src/handlers/ingest.rs` — request
```rust
#[derive(Debug, Deserialize)]
pub struct IngestRequest {
pub title: String,
pub content: String,
pub domain: Option<String>,
#[serde(default)] pub entities: Vec<EntityInput>,
#[serde(default)] pub relations: Vec<RelationInput>,
}
#[derive(Debug, Deserialize)]
pub struct EntityInput {
pub name: String,
#[serde(rename = "type", default)]
pub kind: Option<String>, // wire key is "type" (a Rust keyword)
}
#[derive(Debug, Deserialize)]
pub struct RelationInput {
pub from: String,
pub to: String,
#[serde(rename = "type")]
pub kind: String,
}
Validation constants & helpers (src/handlers/mod.rs)
#![allow(unused)]
fn main() {
pub const DOMAIN_RE: &str = r"^[a-z0-9][a-z0-9_-]{0,62}$";
pub const NAME_RE: &str = r"^[A-Za-z0-9 _-]{1,100}$";
pub const RELTYPE_RE: &str = r"^[a-z0-9_]{1,64}$";
pub const MAX_QUERY: usize = 2_000;
pub const MAX_TITLE: usize = 500;
pub const MAX_CONTENT: usize = 1_000_000;
pub const MIN_LIMIT: u32 = 1;
pub const MAX_LIMIT: u32 = 100;
pub const MAX_ENTITIES: usize = 200;
pub const MAX_RELATIONS: usize = 200;
pub const MAX_BODY: usize = 2 * 1024 * 1024; // 2 MiB — defined but UNUSED; real body cap is the HTTP layer (MAX_REQUEST_SIZE = 1 MiB)
pub const DEFAULT_RECALL_LIMIT: u32 = 5;
pub const DOMAIN_CONFIDENCE_THRESHOLD: f32 = 0.55;
pub fn normalize_domain(raw: &str) -> Result<String, HandlerError>; // → domain_invalid
pub fn normalize_name(raw: &str) -> Result<String, HandlerError>; // → name_invalid
pub fn normalize_rel_type(raw: &str) -> Result<String, HandlerError>; // → relation_invalid
}
provenance(src/search/mod.rs::Provenance) andtelemetry(src/search/mod.rs::SearchTelemetry) are larger structs with nested quality-assessment and PRF-decision types (see §1 / §2 for their serialized field lists). Their full Rust definitions live insrc/search/mod.rsandsrc/search/quality.rs.
7. JSON Schema generation (optional, future)
For a single machine-readable source of truth, derive JSON Schemas from the Rust structs via
schemars (#[derive(JsonSchema)]) and publish them
alongside the OpenAPI spec (openapi.yaml). The TS types can then be code-generated from
those schemas, eliminating manual drift. Noted in ROADMAP Phase 6.
8. Capacity envelopes (v0.9.9)
brain-server publishes a measured (not estimated) capacity envelope per
target hardware. A configuration that exceeds it is unsupported: writes
are rejected with HTTP 507 Insufficient Storage until the operator resolves
it; reads always return 200 (an over-capacity brain must still answer).
| Target | BRAIN_CAPACITY_TARGET | Max docs | Max DB | Max RSS |
|---|---|---|---|---|
| Jetson Nano 4 GB (default) | jetson | 10 000 | 512 MiB | 320 MB |
| Desktop / 16 GB host | desktop | 50 000 | 2 GiB | 320 MB |
/healthreports the live state undercapacity:{ target, docs, max_docs, db_mib, max_db_mib, rss_mib, max_rss_mib, status }wherestatusisok|warning(within 10% of a ceiling) |exceeded.- Writes (
POST /add,/ingest,/ingest/memory,/ingest/markdown) callguard_capacity. Over-capacity →507with body{ "error": "capacity_exceeded: docs=N/M db_mib=.../... rss_mib=.../..." }. - Reads (
GET /search,POST /recall,GET /get/{id}) never check capacity — a brain over its envelope still answers queries. - Tightening for test/constrained deploys:
CAPACITY_MAX_DOCS,CAPACITY_MAX_DB_MIB,CAPACITY_MAX_RSS_MIBoverride the built-in defaults. - Ship gate:
bench --features benchwithBENCH_ENVELOPE=jetsonexits non-zero if RSS or p95 ceilings are breached — turning a measurement into an assertion.
Measured numbers for 1k / 10k / large-vault corpora are published in
BENCHMARKS.md §v0.9.9 (operator step — run on the target hardware).
9. Migration (v0.9.9 — the v1.0 cutover contract)
v1.0.0 splits the single brain.db into per-domain files (global.db +
brain-<domain>.db). v0.9.9 rehearses that cutover without performing it:
the live runtime stays in shim mode (single global DB). The rehearsal proves
the cutover is safe; v1.0.0 executes it.
Per-row migration rule
Every row follows exactly one rule when v1.0 runs the cutover:
| Row kind | Default target domain | Rule |
|---|---|---|
knowledge.domain = 'global' | global | unchanged |
knowledge.domain = '<name>' | <name> | copy to brain-<name>.db; tombstone in global |
sources / source_revisions | follows the linked chunk’s domain | copy with the chunks |
entities / relationships | follows the owning knowledge.id | copy with the chunks |
evidence_links | follows from_chunk_id | copy with the from-chunk |
tombstones | global (audit trail) | never split |
connectors / connector_checkpoints | global (registry metadata) | never split |
audit_events | global (immutable audit trail) | never split |
domain_centroids | global (it IS the routing table) | never split |
webhook_queue | global (transient) | drained before cutover; not migrated |
Rehearsal tool
brain-migrate-rehearse (build with --features migrate) runs the cutover
against a copy of the live DB:
# Stop the server first (WAL must be quiescent).
brain-migrate-rehearse rehearse \
--source ~/.openclaw/workspace/brain.db \
--dest ~/.openclaw/workspace/global.db
Phases: backup (encrypted snapshot via backup::backup) → copy
(VACUUM INTO + run_migration) → verify (row-count + content-hash +
FTS/vec parity + source/revision linkage + evidence_links + audit_events +
schema-version + 50-row vec0 byte spot-check) → report. Exits 0 only when
every check passes; any mismatch leaves the dest file + a precise failure
message. rollback removes the candidate without touching the source.
Recovery (rollback after the v1.0 cutover)
This is the procedure the rehearsal proves is safe:
- Stop the server.
mv brain.db brain.db.pre-v1andmv global.db brain.db(or flipBRAIN_DB_PATH).- Enable
BRAIN_MULTI_DB=truein the launchd plist. - Restart via
scripts/install-service.sh;brain doctorreports v1.0.0. - Rollback if needed: stop server,
mv brain.db.pre-v1 brain.db, unsetBRAIN_MULTI_DB=true, restart. The failedglobal.dbis retained asbrain.db.failed-cutoverfor forensics.
v0.9.9 does NOT perform steps 1–5. It ships the tooling + this contract so v1.0.0 is a rehearsed operation.
v1.0.0 boot-time cutover (automatic)
When BRAIN_MULTI_DB=true is set at server startup, the server performs a
one-shot safety snapshot of the legacy brain.db into global.db:
- Resolves paths via
StorageLayout:legacy_db()(brain.db) andglobal_domain_db()(global.db). - Skips the snapshot if ANY of:
- shim mode (
BRAIN_MULTI_DBoff — the legacybrain.dbIS the global pool); - the marker
~/.openclaw/workspace/.v1-legacy-cutover-doneexists; global.dbalready exists (operator provisioned it);brain.dbhas noknowledgerows (fresh install).
- shim mode (
- Otherwise:
VACUUM INTO '<global.db>'(consistent snapshot, safe under WAL), then writes the marker so restarts never re-copy.
The runtime keeps reading the legacy brain.db for the global domain — the
snapshot exists as a backup the rehearsal tool can verify against, and as the
physical source for any future operator-driven cutover. No data is moved out of
brain.db; the v0.9.x install path is preserved byte-identical.
v1.0 deprecation policy. The legacy /add, /search, /ingest/memory,
and /ingest/markdown routes remain (with Deprecation: version="0.9.5"
header). The primary write path is now POST /ingest; the primary read path is
POST /recall. A future major version may remove the legacy routes after a
deprecation window of at least one minor cycle.
/ingest/memory response (v1.13.3). POST /ingest/memory now returns real
chunk ids: chunk_id (first inserted rowid, null when nothing added),
chunk_ids (all inserted rowids), entries_added, duplicates_skipped, and
status (success|unchanged|error). entry_id is retained as a
deprecated alias of chunk_id (it previously held the count of entries
added, not a usable id). similarity_score: 1.0 is kept as a legacy field.
15. UMP binding (v1.17.3) — Universal Memory Protocol 1.0
The Universal Memory Protocol is the open standard for portable AI agent memory: records carry content hashes and signatures, access is granted by capability tokens, and the same memory moves across servers, agents, and tools. This section is the exact binding brain-server implements. The UMP 1.0 surface is a bounded binding of the spec at github.com/edihasaj/universal-memory-protocol (SPEC.md, wire shape per the actual 1.0 spec, corrected in v1.17.2).
Levels (suite-verified against the reference runner; 13/13, UMP 1.0 / L3)
- L0 — portable-record file binding:
GET /export?format=ump|ump-mdrenders the existing export as UMP records;POST /ingest?format=ump|ump-mdlowers them back (single record or a batch envelope{ump:"1.0", records:[…]}, per-record status, one failure does not abort). - L3 — local integrity layer: with an operator key configured
(
BRAIN_UMP_KEY_DIR,brain ump keygen), records carry the reference §2.8integrity = {content_hash: "blake3:<base32>", signature: "ed25519:<std-base64>", signer: <did:key>}block (v1.17.4 shape — legacy v1.17.3 blocks still verify via dual-read); verify-on-read; capability tokens (§5.2) gate/ump/*+/export. Without a key the server degrades to L2 andGET /ump/capabilitiesreportsconformance: "L2".
GET /ump/capabilities (also mounted as /.well-known/ump.json) is the
§3.1 handshake: {server{name,version}, ump:"1.0", conformance, kinds, bindings:["http","mcp","file"], retrieval_signals, max_recall:50, writable:true, audit:true}.
Routes (non-public except capabilities//.well-known/ump.json)
| Route | Action | Capability verb | Notes |
|---|---|---|---|
POST /ump/remember | Write | write (derive ok) | §3.3 partial record → structured ingest; scope.owner must match principal or be absent |
GET /ump/memory/{id} | Read | read | integrity-verified on read; tampered → dropped |
POST /ump/recall | Read | read | §3.2 {results:[{record, score, signals{…}}]}; same retrieval core as /recall |
POST /ump/revise | Write | write (derive ok) | patch → new chunk + supersession; {id, supersedes:[OLD]} |
POST /ump/forget | Write | write (derive ok) | hard:false soft / hard:true purge; both tombstoned + audited |
POST /ump/feedback | Write | write (derive ok) | outcome followed|overridden|ignored|contradicted → suggest-feedback upsert |
GET /ump/subscribe | Read | read | SSE change feed; {kind,id} events only, never bodies |
POST /ump/audit | Admin | — (denied to tokens) | §9 alias of /audit |
GET /ump/audit/verify | Admin | — (denied to tokens) | §9 alias of chain verify |
Capability tokens (§5.2)
Compact alg.payload.sig (EdDSA) tokens {iss: did, verbs: [read|write|derive|export], scope:{project}, exp} signed by the operator
key; accepted as Authorization: Bearer on /ump/* + /export. Verbs:
reads need read, writes write or derive, export paths export.
Scope must be absent/empty or "global". Expiry enforced at parse
(middleware); verbs × scope at handler entry (cap_gate after authorize).
Unknown/malformed/expired → unauthorized (401).
Redact semantics
exportable:false records are never emitted on non-owner/file paths; PII
redaction ([redacted:…]) applies per the v1.14 principal rules on
/ump/recall and /ump/memory/{id} reads.
§5.3 injection-resistant rehydration (documented obligations)
- Server: verify-before-emit (integrity check before a record is returned) and scope/consent filter before ranking — both are already the recall pipeline order (verify on read; owner scope filter in the SQL).
- Client (documented, not enforced): treat record bodies as untrusted
data — structural framing only, never execute the body, never render
markdown as a command channel. See
SECURITY.md§UMP.