$ ls topics/memory/

memory/

how agents store, retrieve, and lose context

5 articles, newest first·all articles·frameworks/·engineering/

$ cat topics/memory/README.md

Agent memory architecture is three decisions: what shape the store takes, how the right record gets found, and who decides to go looking. The essays here all come out of one production assistant and the memory service behind it. They share a claim you can test against your own build. Storing things was never the hard part. Getting the right thing back at the right moment is.

The reading order

Memory is easy, retrieval is hard sets the problem. Rheo, a headless Claude assistant, had a full conversation ledger, keyword search, and a memory API, and still answered "I don't have context for what you shared". The model does not notice what is missing. So it never calls the tool. The shipped fix was boot-time injection: assemble a bounded memory section into every prompt before the model sees the user's message, for a few hundred milliseconds and one to two thousand tokens a turn.

Give the model the tools, not the context argues the other way. Its question is not how much context to inject but who decides what to retrieve, and its answer is a tool surface over three query modes, episodic, semantic and procedural, with a strict write discipline: write only on a real trigger, handle a conflict by superseding the old record rather than overwriting it. Read it as a June 2026 snapshot of that design, taken before the store grew a vector index. That index arrived later as a derived overlay on the same file; Vectors in the file you already have, over on the engineering hub, is where it gets built, and why letting it fail costs recall and not correctness.

Those two do not resolve into one answer, and the corpus does not pretend they do. One is a production case study where model-invoked retrieval failed and push shipped instead. The other is the architectural argument for pull. Hold both.

One call, the real prerequisite explains what makes either of them work. Rheo's session boot went from four memory round-trips to one, and the essay settles why: the store's four shapes were already orthogonal enough for one call to answer all four. Cutting call count is downstream of store design, not a bundling trick you can apply to a store that is already tangled.

The workers that keep a structured memory graph clean is the capstone and the most implementation-heavy piece here. It bets on a memory graph with exact lookup by name and structure instead of vector similarity, keeps embeddings underneath as a derived index that can fail without anything being lost, and settles the maintenance question with four nightly workers in a fixed order, on one rule: the model identifies, deterministic code executes, and the model never writes to the graph itself.

What this pillar is not

Memory covers what an agent retains and retrieves, and how each memory layer is shaped. It stops at how several agents get sequenced and gated around that store, which is frameworks, and at how the store is run in production, which is engineering: which database, what it costs, whether the index survives a bad night. An essay here will name the storage tech in passing. The operational answer lives on the other hub.

Where to start

Read Memory is easy, retrieval is hard first. It names the failure everything else in this pillar responds to. Then One call, the real prerequisite if you are still designing a store, or The workers that keep a structured memory graph clean if you have one and it is drifting. Once several agents are reading and writing that store, the sequencing question is frameworks.

$ ls -lt topics/memory/

· memoryOct 2026True of a dateFive essays described my old agent memory store. The module that replaced it is a record of the order I built it in, not a ruling on graphs versus vectors.12 min · memoryJul 2026How I built the workers that keep a structured memory graph cleanRecallatron keeps agent memory as an entity graph and episodic timelines, retrieved by name and structure rather than by vector similarity. Four nightly workers handle resolution, deduplication, profiling, and auto-confirmation. The graph stays usable without a human doing maintenance.10 min · memoryJun 2026Give the model the tools, not the contextThe naive way to give an AI assistant memory is to inject recent turns into every prompt, and it hits a wall fast. The better design hands the model a set of tools and lets it decide what to retrieve, what to compress, and what is worth writing down at all.8 min · memoryJun 2026One call. The real prerequisite.Rheo's session boot went from four memory calls to one. The call count was never the achievement; the store's shape was.6 min · memoryJun 2026Memory is easy. Retrieval is hard.We assumed memory would be the hard part of building Rheo, our Claude assistant. It wasn't; the bottleneck was retrieval, not storage.6 min

$ cat glossary/memory/

agent memory: Agent memory is an AI agent's capacity to retain and retrieve information from prior turns and sessions.
boot-time injection: Boot-time injection assembles a fixed memory section into every prompt before the model sees the user's message.
context engineering: Context engineering is the practice of deciding what goes into a model's context window at runtime: what gets retrieved, what gets injected before the model reads anything, what gets compressed, and what stays out.
context stuffing: Context stuffing crams raw history and documents into a model's context window without curation.
entity resolution: Entity resolution identifies which raw extracted facts refer to the same real-world thing and mints a canonical node for it, turning loose sentences into a Person or a Project.
episodic memory: Episodic memory is an agent's record of what happened and when, kept as short digests of past sessions and looked up by recency or keyword rather than by entity.
memory graph: A memory graph stores an agent's entities and relations for exact lookup by name or graph traversal rather than vector similarity.
memory layer: A memory layer is one store inside an agent's memory system, with its own shape, its own lookup method and its own lifetime, such as a conversation ledger, a digest table or an entity graph.

all terms