Skip to content

Architecture

Every station on this page names the files it describes. The margin is where to look.

Every box here is a file you can open.

There are two paths through this system and they meet in one database. Along the first, something that arrived becomes claims that are filed, versioned and kept when they stop being true. Along the second, a question becomes a plan, six searches, one fused set, and an answer that says what it used. A third path leaves the building entirely and asks a person for permission first. And when the question is a job rather than a recollection — find this, put it there, tell them — a fourth path plans it as steps, runs them in waves, waits for what answers later, and has a second model refute the reply before a person reads it.

Nothing on this page is aspirational. Where the product does not do a thing, it is not drawn — and the last section names the things it does not do, because those are what people find out the hard way.

The path

From something arriving to an answer that cites it.

Twelve stations in two bands. The top band runs when material arrives; the bottom band runs when somebody asks a question, which is usually hours or months later. They are drawn apart because they are apart: they share a database and never call each other, and a single line from one end to the other would imply a request goes in and an answer comes out — the most misleading thing a diagram of this system can say.

  1. 01 — Arrive

    In
    a message, a file, a transcript, a call
    Out
    a raw input, with the source that produced it

    Material turns up in whatever shape it is already in.

    A Telegram message, a forwarded email, a file in Drive, a Meet transcript, an upload, a call from the MCP server, the CLI or an SDK. Nothing is tagged and nothing is filled in by hand. A surface that has no credentials configured is not mounted at all, so a route that would always fail does not exist to be found.

    • packages/ingestion/src/connectors/telegram/connector.ts
    • apps/api/src/integrations/chat-platforms.ts

  2. 02 — Normalise

    In
    a raw input
    Out
    a normalised document: text, blocks, participants, provenance

    Whatever arrived becomes one kind of document.

    Everything downstream reads a single shape. The flat text is kept for reading and the blocks are kept for structure, each block carrying an offset into the original — which is what later lets an answer point at a page rather than at a document. Audio is transcribed. A scan with no text layer goes to OCR as its own job, because a single failure there must not re-run everything else.

    OCR is by an order of magnitude the slowest work here, which is why it has its own queue and its own ceiling.

    • packages/ingestion/src/normalization/normalized-document.ts
    • packages/ingestion/src/normalization/blocks.ts
    • packages/async/src/jobs/kinds.ts

  3. 03 — Extract

    In
    a normalised document
    Out
    candidate memories, with evidence pointing back into the document

    Claims are pulled out, and what said them is kept.

    Not keywords: statements somebody could disagree with, each typed as one of nineteen kinds — a decision, a commitment, a preference, an assumption — and each carrying a confidence, an importance, and the evidence it came from. One paragraph may produce several, or none. That is why writing answers 202 with a job id rather than a memory.

    A write cannot be read back immediately. The endpoint returns the job to poll rather than pretending otherwise.

    • packages/memory/src/extraction/memory-extractor.ts
    • packages/async/src/handlers/extract-handler.ts
    • packages/db/src/schema/candidates.ts

  4. 04 — Reconcile

    In
    candidate memories
    Out
    memories, versions, supersessions and conflicts

    It meets what is already known, and nothing is overwritten.

    The same person named three ways becomes one entity. A fact already held is corroborated rather than stored twice. A fact that replaces an older one supersedes it — the old memory keeps its dates and its sources and moves to the superseded state, from which it can be archived or deleted and never restored. A fact that merely disagrees becomes a conflict holding both sides.

    A sequence is not a disagreement. Used AWS until June and uses GCP since July is a change, and flagging it would train people to ignore conflicts.

    • packages/memory/src/consolidation/consolidation.ts
    • packages/memory/src/conflicts/conflict-detection.ts
    • packages/memory/src/lifecycle/supersession.ts

  5. 05 — File

    In
    a memory
    Out
    space_memberships rows

    A memory is filed into Spaces by a row, not moved into a folder.

    A Space is a filing rule, an audience and a retention policy — not a container. Membership is a link row, so one memory sits in several Spaces at once and a correction reaches all of them. There is deliberately no membership row for Universal: Universal is the base layer, and a row per memory saying so would carry no information. A conversation carries its own Spaces, and the memories extracted from it inherit them, because the scope of a conversation is known when it starts. Removing a memory from a Space ends the row rather than deleting it, so a re-add is visibly a re-add.

    A Space can be shared with another person, and that is visibility rather than ownership: nothing is copied into their account, so revoking it removes the memories from their next query and leaves no orphaned copy.

    • packages/db/src/schema/space-memberships.ts
    • packages/memory/src/spaces/space-filing.ts
    • packages/db/src/schema/space-collaborators.ts

  6. 06 — Embed

    In
    a memory
    Out
    a vector, in the same database as the memory

    The vector is written last, and separately.

    Embedding is its own job, after the memory exists and after everything that gives it meaning. Changing embedding model is then re-embedding in the background rather than a migration that stops writes — and a deployment with no vector store at all is a deployment whose questions are answered by the other five strategies rather than one that fails.

    Same database on purpose: a memory and its embedding in two systems with no transaction between them drift apart, and nothing notices.

    • packages/async/src/handlers/embed-handler.ts
    • packages/db/src/schema/vectors.ts

  7. 07 — Ask

    In
    a question, and who is asking
    Out
    an intent with a confidence, and explicit constraints

    The question is read before anything is searched.

    Eleven intents, decided by deterministic patterns rather than a model call. What did we decide wants a claim; how did this change wants a sequence; show my tasks wants a filter and no embedding at all. Separately, what the person actually said is taken as constraints — from Slack, in this Space, decisions only. An intent is a guess and boosts; a constraint is an instruction and filters. Collapsing the two lets a guess hide the answer.

    • packages/context/src/query/intent.ts
    • packages/context/src/query/constraints.ts

  8. 08 — Plan

    In
    an intent and its constraints
    Out
    a retrieval plan, and a token budget

    Which retrievers run is decided before any of them do.

    The plan is a value: the steps, a limit and a weight for each, how many survive to assembly, whether to spend a model call reordering them, whether to look for contradictions, and a sentence saying why. Because it exists before execution, a slow or a disappointing answer can be explained afterwards. A weak intent guess widens the plan rather than narrowing it — being unsure is a reason to cast a wider net.

    • packages/context/src/planner/planner.ts
    • packages/context/src/planner/plan.ts
    • packages/context/src/planner/strategies.ts

  9. 09 — Retrieve

    In
    a plan
    Out
    results per strategy, and a list of the ones that failed

    Six strategies run at once, and a failure is a value rather than an error.

    Semantic, structured, temporal, relationship, source and space. They touch different indexes and share nothing, so running them in sequence would make a query as slow as the sum of its parts. One failing does not fail the question: it reports itself as failed, and the answer says part of the retrieval did not complete rather than claiming nothing was found. A deployment with no vector store simply has no semantic step.

    • packages/context/src/retrieval/engine.ts
    • packages/context/src/retrieval/ports.ts

  10. 10 — Fuse

    In
    results per strategy
    Out
    one set, each memory carrying every strategy that found it

    Agreement between strategies is evidence, and is scored as such.

    A cosine similarity of 0.82 and a structured match of 1.0 do not measure the same thing, so each strategy is normalised within itself before anything is combined, then weighted by what the plan said it was worth for this question. A memory found by similarity, by an entity filter and by a graph walk is three independent systems reaching the same conclusion, and gains a bonus that is additive and saturating — so three weak agreements never outrank one exact match.

    • packages/context/src/fusion/fusion.ts
    • packages/context/src/fusion/provenance.ts

  11. 11 — Assemble

    In
    a fused set
    Out
    an ordered window, with what did not fit counted

    Ranked on ten separate signals, then cut to what fits.

    Retrieval score, entities named, how the validity window fits the question, recency, the memory's own confidence and importance, how many independent sources back it, nearness in the graph, how many strategies agreed, and whether you pinned it — kept as ten values rather than one opaque number, because when the wrong memory ranks first the only useful question is which signal put it there. Room is reserved for the conflicts before the memories are packed, so a long list cannot crowd out the thing that says the record disagrees with itself.

    • packages/context/src/ranking/signals.ts
    • packages/context/src/ranking/scoring.ts
    • packages/context/src/assembly/window.ts

  12. 12 — Answer

    In
    an ordered window
    Out
    text a model can be given, with citations and what is missing stated

    It says what it used, and what has stopped being true.

    Contradictions are printed first, because a model that reads two confident statements and only afterwards learns they disagree has already committed to the first. Current memories are separated from historical ones under a heading that reads NO LONGER TRUE — these WERE true and have since changed, because an unlabelled superseded memory is read as current fact. Each memory is numbered by its position and labelled with the source it came from. A truncated window, a search that half failed, and a genuinely empty memory are three different sentences and never the same one.

    • packages/context/src/assembly/render.ts
    • packages/context/src/assembly/citations.ts

Station 10 — in depth

Agreement between searches is evidence.

A memory that vector similarity liked, a filter matched and a graph walk reached is three independent systems arriving at the same conclusion — much stronger than any one of them scoring it highly, and a fusion that simply took the best score would throw it away. Each search is normalised within itself first, because a cosine similarity of 0.82 and a structured match of 1.0 do not measure the same thing; then agreement adds a bonus that saturates, so three weak agreements never outrank one exact match.

010203

Seven memories, three searches. Two were returned by every one of them.

The sheets, top to bottom

  1. 01 · semantic

    vector similarity — memories that sound like the question

  2. 02 · structured

    filters — type, state, Space, provider, source, confidence floors

  3. 03 · relationship

    a graph walk out from the entities the question named

Three more exist and are planned per question rather than always run: temporal, source and space. A question about the past puts temporal first, because when something was true is the question, and similarity is only how the thread is found within it.

packages/context/src/planner/strategies.ts

Reach

The path that leaves the building.

Memory is worth little if answering is all it can do, so the system can act on somebody’s own machine — and that is the one capability where every step exists to slow it down. An agent runs on the laptop and dials out; nothing ever calls in. A command is proposed, shown as the exact argument list that will run, approved by a person against that line, and then judged a second time by the machine itself against rules the server cannot change.

  1. 01 — Propose

    In
    an ask, from a chat or the Ask page
    Out
    an agent_requests row, awaiting approval

    A command is proposed. Nothing has run.

    Asking for a command writes a request row and stops. The tool's own reply says NOTHING HAS RUN YET in as many words, because a model that reports a command as executed when it is queued is the failure that makes the whole gate decorative.

    • packages/tools/src/catalogue.ts
    • apps/api/src/services/agent-service.ts

  2. 02 — Show

    In
    a proposed argv
    Out
    one line, shown identically on the page, in the email and in the CLI

    The line a person reads is derived from the list that will run.

    The argument list is stored as a list and never as a string, so a semicolon, an ampersand or a backtick is a character inside an argument rather than an instruction. The human-readable line is rendered from that list in one place. Two independently supplied fields could disagree, and then somebody approves one command while another runs.

    • apps/api/src/services/agent-service.ts
    • packages/db/src/schema/agents.ts

  3. 03 — Approve

    In
    a request, and a person looking at it
    Out
    an approval against that exact line, or a refusal

    A command always waits for a person, whoever asked for it.

    A request a model caused starts in a state the machine's claim query cannot see, so it is not merely unapproved — it is invisible to the agent until a person acts. Everything this system reads is material anybody could have sent: a tool call is never on its own evidence that somebody wanted a command run on their own disk. There is no flag that waives this and no surface that skips it.

    • apps/api/src/services/agent-service.ts
    • apps/web/app/(app)/dashboard/requests

  4. 04 — Judge

    In
    an approved argv
    Out
    a run, or a refusal that says which rule refused it

    The machine judges it again, against its own rules.

    The agent dials out from the laptop and asks whether there is anything for it; nothing reaches in, so there is no port to open and no server on that machine to secure. What it claims is judged locally by what the command does rather than by its name — reads, writes, reaches the network, or interprets a language. Anything that reaches the network is refused, and so is anything that runs a language, whatever was approved: private data plus untrusted content plus a way out is exfiltration, and no approval dialog catches it, because the person approving cannot see where the bytes go. There is no shell. Paths are confined to roots given on that machine's own command line, resolved through symlinks on both sides.

    The allow list is not the boundary. A list of program names cannot express what a command does, and treating it as the control refuses a build somebody wanted while permitting a recursive read of the whole disk.

    • packages/cli/src/commands/run-command.ts
    • packages/cli/src/commands/agent.ts

  5. 05 — Deliver

    In
    what the command printed
    Out
    a reply in the same chat, page or terminal that asked

    The output goes back to the surface that asked, and nowhere else.

    Output is capped and the command is stopped if it overruns either the size or the time bound — and both facts are printed above the output, because a log that was silently cut is worse than one that was refused: the reader reasons about the fragment as though it were the whole of it. The environment is stripped before the command runs, because that process holds a credential.

    • packages/cli/src/commands/run-command.ts
    • packages/cli/src/commands/agent.ts

The engine

Twenty step kinds, counted from packages/engine/src/capabilities.ts.

A job is planned, worked in parallel, and checked before it is reported.

Eight stations in two rows around one record. It was built to work the way several people work on one problem at once: somebody plans, several work at the same time on the parts that do not depend on each other, one waits for the answer that comes later, another reads the draft and says what it claims that nothing supports, and the report names what did not happen as plainly as what did. Every worker is a bounded model loop with only its own kind’s tools, and anything that leaves the building still takes the path above and waits for a person.

Memory sits under all of it: a step whose answer is already held is answered from memory and says so, what a run finds feeds memory back, and a failure asks one question whose answer is kept for the next time. What it can do, in full.

  1. 01 — Record

    In
    a message, and the conversation it is in
    Out
    the last turn's record, or nothing

    The last turn is read before anything is planned.

    Every turn on every surface begins a row and settles it with what it planned, what each step did, what evidence it had, what it asked and what was still running when it wrote. The next message in the same conversation reads that row first, so “the first one” means the first of the list that arrived, and a step still waiting is not started again.

    • packages/db/src/repositories/turn-repository.ts
    • packages/db/src/schema/assistant-turns.ts

  2. 02 — Plan

    In
    the message, the thread, what is connected, what worked before
    Out
    a plan, a question, or “just answer”

    A question becomes steps with dependencies, or one question back.

    A reasoning model is told what this system can do — twenty kinds of step, each with its tools and the argument names a plan may carry — and answers with steps that name what they need from one another, or with one question when two things could be meant. A one-step ask takes the ordinary loop. Nothing in a plan is a value the planner invented: a path, an address or a name nobody gave is left out, and a step missing one asks.

    • apps/api/src/services/answer-planner.ts
    • packages/engine/src/capabilities.ts

  3. 03 — Known

    In
    a step and its words
    Out
    an answer with a memory id, or the step untouched

    Memory answers a step before a worker is sent.

    Before a step runs, what memory holds is checked against it: a fact of the right kind, recent enough for that kind, from a memory or from a step answered in an earlier run. A step answered that way is done with the memory named as its evidence, so the reply can say where it came from. Four refusals: never a step that acts, never a question about now, never past the age that kind allows, and never silently.

    • packages/engine/src/already-known.ts
    • packages/engine/src/known-for.ts

  4. 04 — Work

    In
    the plan and the record so far
    Out
    an outcome per step

    Workers run in waves, each with only its kind's tools.

    Steps whose needs are met run together; the next wave starts when they settle. A worker is a bounded model loop offered its kind's tools and nothing else — a find worker cannot send mail — and it answers in one agreed shape: done with evidence, pending, failed with a reason, or a question. Every path, id and address it passes must come from the person or a tool result. Anything that leaves the building goes through the desk on the path above and waits for a person.

    Bounded. How many rounds a worker gets, and whether it thinks at all, is configuration — per deployment and per person.

    • packages/engine/src/answer-plan.ts
    • packages/engine/src/step-agent.ts
    • packages/engine/src/answer-workers.ts

  5. 05 — Wait

    In
    a pending step, and an answer that arrives
    Out
    the rest of the plan, run

    A step that answers later leaves the plan waiting, and the answer resumes it.

    A computer that is asleep, a video that takes minutes to fetch and transcribe: the step reports pending, the steps after it wait, and the reply says so rather than calling it a failure. When the answer lands — the machine's file, the worker's finished video — the turn is resumed with the steps already done given, so nothing runs twice and only what was waiting runs.

    • apps/api/src/services/chat-service.ts
    • packages/async/src/handlers/video-handler.ts

  6. 06 — Review

    In
    the draft and the record
    Out
    a reply the record supports, or an honest failure

    A second model refutes the draft; the rules keep the final say.

    The reply is drafted from the record, then a reasoning model is handed the message, the record and the draft as data and asked only for what the draft claims that the record does not support. Its problems feed one rewrite. Then deterministic rules check the result: no path, id or address nobody reported, no “sent” without evidence of a send, no caveat dropped. A reviewer that fails is a warning, never a closed gate.

    • packages/engine/src/answer-reviewer.ts
    • packages/engine/src/answer-verifier.ts

  7. 07 — Report

    In
    the settled record
    Out
    the reply, and the record written back

    The reply names what happened and what did not.

    Written from the settled record, in the person's own terms: what was found, what is still waiting and where it will arrive, what failed and why in the tool's own words, and the one question when a step needed one. A run that ended in a question puts the question as the reply, with nothing else pretending to be progress.

    • packages/engine/src/report-run.ts
    • apps/api/src/services/chat-service.ts

  8. 08 — Learn

    In
    outcomes, and the person's next message
    Out
    evidence rows, expiries, and a remembered resolution

    A run feeds memory, and a failure asks one question and keeps the answer.

    What a run found supports the memories it agreed with and expires the ones it contradicted — the same importance and evidence machinery the extractor uses, so nothing has a second notion of truth. A correction in the next message contradicts the step it corrects, at once. A turn that ends in failure asks one question; the next plain message is kept as the answer and handed to the planner the next time a request like it arrives — as advice in the person's words, never as a value.

    • packages/engine/src/run-feedback.ts
    • apps/api/src/services/feedback-memory.ts
    • apps/api/src/services/question-learning.ts

Decisions

Five decisions, and what each one costs.

The cost is stated, because a list of decisions with no costs attached is a list of opinions.

Writing is asynchronous, and says so

Capturing something answers 202 with a job id and the sentence “it becomes memory once extraction and consolidation run, which may produce one memory, several, or none”. Extraction, entity resolution and conflict detection all happen after the response.

The cost. A write cannot be read back immediately, which is the first thing everybody trips over. Doing it inline would mean a request that takes as long as a transcription, so the endpoint hands back the job to poll rather than pretending.

apps/api/src/routes/memory.ts

Retrieval degrades, it does not fail

A strategy that fails reports itself as failed and the others still answer, with the result marked degraded. A deployment with no vector store has no semantic step rather than no search.

The cost. A client that ignores the flag reports search as broken when it is merely narrower — and, worse, reports nothing when it silently is. The answer text says so too, for the model that will read it.

packages/context/src/retrieval/engine.ts · apps/api/src/services/search-service.ts

Contradictions are kept, not resolved

When a new claim disagrees with a stored one, both are kept with their sources and dates. A replacement supersedes; a disagreement becomes a conflict holding both sides, and a superseded memory can be archived or deleted but never restored — coming back requires a new memory superseding the one that replaced it.

The cost. The record grows, and questions about the present have to exclude the past explicitly. Silently overwriting would make the system confidently wrong at exactly the moment it should be uncertain, with nothing left to explain why it changed its mind.

packages/memory/src/core/memory-state.ts · packages/memory/src/conflicts

A Space can never widen a memory

Filing is checked against the Space's own audience: a Space no wider than the memory adds no readers and is allowed, a wider one is refused. Sharing a Space with somebody grants visibility, never ownership — nothing is copied into their account, so revoking it removes the memories from their next query and leaves no orphaned copy behind.

The cost. Organising can be refused, which is a worse moment than it sounds. The alternative is that tidying your notes quietly becomes a disclosure.

packages/memory/src/spaces/space-scope.ts · packages/db/src/schema/space-collaborators.ts

Counters are exact; the request log is not

Request counts come from rollups that count everything. The per-request log is sampled — five per cent by default — and failures are kept in full whatever the rate.

The cost. The drill-down cannot be totalled, and the two must never be read as one number. Counting sampled rows reports a fraction of the traffic and reports it confidently, which is worse than reporting nothing.

packages/db/src/schema/observability.ts · apps/api/src/observability/sink.ts

Underneath

What it runs on.

Postgres, pgvector
Memories, their versions, entities, relationships, evidence, conflicts, Space membership and the vectors are one database. A memory and its embedding in two systems with no transaction between them drift apart, and nothing notices until an answer cites something that no longer says that.
Redis
The job queue and the shared rate-limit counters. Without a shared store the configured limit is silently multiplied by however many processes are running, and nobody can say what the real one is.
Eight job kinds
Ingest, OCR, extract, consolidate, embed, event, maintenance and video — each with its own timeout and its own concurrency, because what a job contends for differs per kind and is invisible from the queue. A kind is added when a handler exists to run it, never before; video is the newest, and the one whose answer resumes a waiting plan.
Several model providers
No call site names a vendor. Every call asks the router for a capability — chat, tool calling, reasoning, extraction, vision, transcription — and one map says which models serve each, in what order: Groq leads text and tool rounds with NVIDIA, Cloudflare and OpenRouter rotated beside it, NVIDIA leads reasoning, and OpenAI is held back as the paid last resort. Calls rotate across accounts within a vendor, and the fallback reaches every vendor in the chain before the paid tier, so one vendor being unavailable is a slower answer rather than an outage. Retrieval intent is classified deterministically; the planner that turns a job into steps is a reasoning call, and a planner that fails hands the message to the ordinary loop rather than to nobody.
Blob storage
Original files. References travel through the queue as keys and never as signed URLs, because a URL with a grant baked into it outlives the grant once it is sitting in a dead-letter row.

Not drawn

Three boxes that are missing on purpose, and one that is smaller than it looks.

Each of these is a thing people reasonably assume is here. None of them is, and finding that out from a diagram is cheaper than finding it out from a failed integration.

  • No GitHub connector

    There is no GitHub integration. The connectors that exist are Telegram, email in and out, Google Drive, Gmail, Meet transcripts, and the MCP server — plus Slack, WhatsApp and Microsoft Teams, which are written and mounted only where credentials are configured, because a route that exists and always fails cannot be told from a broken one.

  • No source is watched for you

    Nothing monitors a source and tells you when it changes — no server ping, no repository poll, no alert you can define. Material arrives because something sent it or a sync ran. The scheduler does run on a timer, and it does more than housekeeping: besides sweeping expired candidates and draining the outbox, its five-minute tick reads task deadlines up to sixty days ahead and emails you about them. So the honest line is that the system speaks first about deadlines it already holds, and about nothing it went and looked at.

  • Tasks have no screen yet

    Task is one of the nineteen memory types and tasks are extracted, stored and reachable through the tools. There is no page in the application that lists them, so this page does not draw one.

  • The answer loop sends nothing a person did not confirm

    The loop that answers a question can now send mail, forward a message, mail a file off a computer, put a file onto one, file into a Space or share one — but every one of those is a proposal, not an act. The exact act is rendered in the chat — the file, its full path, the machine, who it is going to — and it runs only when the person types a single-use code into that same conversation. On a surface that cannot show them that, the tool is not offered at all. Retrieved memory — assembled from material anybody can write into by sending an email — therefore never reaches a way out without a person reading the act first. The machine agent still refuses any command touching the network whatever was approved: private data, untrusted content and a way out are the three things that together make exfiltration, and no approval dialog catches it.

Next