The naive version of giving a chatbot memory looks obvious. You keep a buffer of the last dozen messages in process, you feed it back on every turn, and the model sounds like it remembers the conversation. Then the process restarts and the whole thing is gone. Every restart is total amnesia. The assistant can have its open tasks handed back to it, but ask it "what did we decide about that project last week?" and it has nothing, even though the conversation happened.
So you try the obvious fix: persist the turns, and inject the last N of them on every prompt. This works for a while. Then it hits two walls, fast. The first is tokens. Dumping recent history into every call gets expensive, and most of what you inject has nothing to do with the question on the table. The second is coherence. Past roughly twenty turns the signal-to-noise ratio drops and the model starts losing the thread. And the thing you actually wanted, the decision from last week, fell out of the window days ago. It isn't in the buffer to inject. The failure is quiet, which is the worst kind: nothing errors, the model just doesn't know, and you can't tell from the outside whether something was never discussed or simply scrolled out of range.
The better question
The mistake is in the question. "How much recent context should we inject?" assumes the harness has to decide, up front, what the model will need, before the model has even read the prompt. It can't know that. The better question is "who decides what to retrieve?" and the answer that holds up is: the model does, on demand, after it has seen what you asked.
That's the whole shift. Instead of the harness pre-loading a fixed window and hoping it guessed right, you hand the model a set of tools and let it pull what it needs when it needs it. This isn't a new idea. MemGPT, now Letta, made the case a few years back: let the model page context in and out with tool calls, the way an operating system pages memory. Mem0's fact extraction, Zep and Graphiti's temporal graph, the graph-retrieval line of work, they all point the same way. The agent should manage memory, not just consume whatever it was handed.
What the tool surface looks like
Three query modes, one store
Once the model is doing the retrieving, you have to ask what it retrieves from. It helps to stop treating "memory" as one thing. There are three different questions an assistant asks of its past, and they want different storage and different lookups.
Episodic: what happened, and when. The raw turns, plus short digests of past sessions. You look these up by recency or by keyword.
Semantic: what's true about the world. People, projects, deadlines, preferences. This is a graph of entities and a table of facts. You look them up by name with a substring search, or by walking from one entity to its neighbours.
Procedural: how the user likes things done. Recurring patterns, templates, routines. You look these up by keyword or category.
The part worth noticing is that these are not three separate systems. They are three query modes over related data. A single turn can drop a record into all three at once: an episodic note that the turn happened, a semantic update because the user named a new deadline, and a procedural refinement because they corrected how they want something formatted.
The layers
Underneath the three modes sits a stack of layers. I'll describe them at the architecture level, not the schema level, because the shape is the point.
Layer 0 is the ledger: an append-only log of raw turns. Nothing is deleted or updated in place, ever. A keyword full-text index (FTS5) sits on top so you can search the text. There is no embedding column. Search here is keyword, not meaning.
Layer 1 is digests. At a session boundary (a gap of a couple of hours, or an explicit signal that the session is done) one model call runs over the session's turns and produces a three-to-five sentence summary, plus candidate entities and procedural notes it noticed. Candidate is the key word. They are proposed, not trusted. They sit in a reviewable state until something confirms them.
Layer 2 is topic threads: recurring concerns pulled out of the digests and linked back to the sessions they showed up in. These summaries are built structurally, with no model call.
Layer 3 is the entity graph. It is an append-only JSONL file: entity records, plus confirm and supersede patch records written into the same file, never rewritten in place. Each entity carries where it came from, a confidence score, a valid-from and valid-until, and an optional pointer to whatever superseded it. Relations live inside an entity's properties, sparsely, rather than in a separate edge table. It is a log of facts and corrections, not a mutable state you overwrite.
Layer 4 is the tool surface: a set of tools, over JSON-RPC, that the model calls to read and write every layer below.
The tools the model actually calls
In practice the model reaches for a small handful of these.
chat_search runs a keyword search over the turn ledger. One mode, keyword. No vector search, no hybrid ranking. If you search for a word the conversation never used, you won't find the turn. That's a real limit, and I'll come back to it.
entity_search does a substring search over the entity graph: find the project, the person, the deadline by name.
memory_context is the boot bundle. On startup it returns a small, bounded payload: a few topics, a few entities, the confirmed procedural notes, the last several memory items, the open tasks. Not the whole store. A light starting context, and then the model pulls more with the other tools if it turns out to need it.
summarize_and_archive is how the model triggers compression itself, at a session boundary, instead of waiting for the harness to do it on a timer.
The pattern across all of them is the same. The model decides when to search, when to compress, and what to surface.
The harness stopped guessing.
The write discipline
Retrieval is only half of it. The other half, the half that decides whether any of this is worth trusting a month in, is what the model writes down. A memory store is only as good as what's in it, and the fastest way to wreck one is to save everything.
When to write
The model calls write_memory on a short list of triggers. A new person, project, or deadline mentioned for the first time. A deadline, preference, or standing fact stated or confirmed out loud. A correction, where the user says the thing on file is wrong. That's close to the whole list. These are deliberate events, not "store anything that seemed interesting."
When to refuse
The refusals matter as much as the writes. The model doesn't record conversational pleasantries, speculation, or anything the user flagged as likely to change. And it does not copy in things that already live in the task or ticket system, because that record is the canonical one, and duplicating it into the memory graph just creates two sources of truth that will disagree later.
This restraint is designed, not accidental. A graph that fills up with noise stops being trusted faster than it earns trust, and once you stop trusting it you stop reading it, and then it's dead weight. What the model refuses to remember is specified as carefully as what it keeps.
Conflicts: version, don't overwrite
The interesting case is when a new fact disagrees with an old one. The cheap answer is to overwrite: new value wins, old value gone. But this system doesn't do that. Every fact carries its source and an optional pointer to whatever replaced it, so nothing is lost silently. The handling depends on the kind of conflict:
- Same fact, minor wording difference: consolidate quietly and log it in the entity's history.
- Same fact, different value (a deadline moved): keep both. The old one gets a superseded-by pointer, the new one lands at around 0.8 confidence.
- Two high-confidence facts that flatly contradict each other: keep both and flag it for review. The model does not auto-resolve a high-confidence contradiction. It isn't allowed to decide, on its own, which true-sounding thing is the real one.
- The user explicitly corrects something: supersede it, keep the old version, set the new one to full confidence.
- Low-confidence candidates, the ones extracted from chat but never confirmed, can be quietly revised.
The tool that does this is entity_supersede. It doesn't delete the old record. It writes a new one and points the old at it with a timestamp. The graph ends up as a versioned history of what was believed and when, not a single cell you keep painting over. Ask it something later and it can tell you the current answer and how the answer got there.
What isn't built yet
It's worth being plain about the edges, because a memory system that oversells itself is worse than one that admits where it stops. Search today is keyword and substring only. There is no embedding column, no vector search, no hybrid retrieval: ask for a concept in words that never appeared in the stored text and the lookup misses. Proactive surfacing, the assistant noticing on its own that a fact has gone stale and raising it unprompted, isn't built. Neither is pulling memory in from other channels, like email. There's no decay scoring on time-sensitive facts; the only thing holding rot down today is a nightly job that prunes stale, unconfirmed, low-confidence entities. And there are no formal recall or false-write benchmarks yet, so the quality claims are reasoned, not measured. All of that is future work, named honestly so the present doesn't get oversold.
What changes
So here is the concrete difference, the part the user actually sees, setting the architecture aside.
Ask the assistant "what did we decide about that project?" and it doesn't sift through whatever happened to land in the injection window. It calls a tool, searches the turns or the entity graph, and comes back with an answer that carries its source and a timestamp, so you can see where it came from and when. A restart stops being amnesia, because the durable facts are in the graph and the model knows how to go and get them. And the memory grows as the conversations grow: every session that ends adds turns to the ledger, a digest to the record, maybe an entity or a corrected fact to the graph, all of it sitting there to be retrieved the next time the model decides it needs it.
The memory system described here is The Recallatron, a persistent memory engine for AI agents. More at recallatron.com.