The query that finds nothing
M.O.T., the memory store behind my assistant Rheo, had one way to search for most of its life: FTS5, SQLite's built-in full-text index. FTS5 is fast and it doesn't lie to you, but it only matches words. Ask for "the school thing" and chat_search("the school thing", mode:"fts") comes back empty. The turn it should have found says "Lincoln Elementary." No token from the query appears in the stored text, so FTS5 returns nothing. That is exactly correct behavior, and it is exactly useless here.
Porter stemming doesn't rescue this. Stemming folds "running" down to "run"; it does nothing when the query and the answer share no root at all. The user is reaching for the meaning, not the words, and the meaning is what a keyword index can't see. Closing that gap was the whole of Track 5, and it needs two pieces: an embedding model, fastembed, and a vector store, sqlite-vec.
Same connection, same file
An earlier piece in this series, "one process, one file," said it plainly: "No vector service." Adding semantic search looks like it breaks that promise, because the usual way to get vectors is to stand up a vector database next to your real one and keep the two in sync. Track 5 doesn't do that.
sqlite-vec is an extension, not a service. It loads into the same SQLite connection handle that already serves everything else, and its tables live in the same file. Version 0.1.9 ships prebuilt binaries per platform through npm optional dependencies, so no compiled .so sits in the repo; the package exports getLoadablePath(), and a SQLITE_VEC_PATH env var overrides it when auto-resolution fails.
The one structural constraint is boot order. CREATE VIRTUAL TABLE ... USING vec0(...) needs the extension already loaded, so loadVecExtension(db) has to run inside getDb() before any migration. That reordered the boot sequence in the db client. Migration 0007 then creates four virtual tables, each with a declared primary key the upsert and prune paths later need as a delete target:
CREATE VIRTUAL TABLE IF NOT EXISTS conversation_vec
USING vec0(turn_id INTEGER PRIMARY KEY, embedding FLOAT[384]);
CREATE VIRTUAL TABLE IF NOT EXISTS memory_items_vec
USING vec0(item_id INTEGER PRIMARY KEY, embedding FLOAT[384]);
CREATE VIRTUAL TABLE IF NOT EXISTS entity_vec
USING vec0(entity_id TEXT PRIMARY KEY, embedding FLOAT[384]);
CREATE VIRTUAL TABLE IF NOT EXISTS session_digest_vec
USING vec0(session_id TEXT PRIMARY KEY, embedding FLOAT[384]);The embeddings come from fastembed v2.1.0, running all-MiniLM-L6-v2 at 384 dimensions. It earns its place for boring reasons. M.O.T. runs as a Next.js 14 App Router server in CommonJS, and fastembed ships a dual CJS/ESM package, so it imports with no config changes; the pure-ESM alternatives would have needed module-type surgery. It runs native CPU inference through onnxruntime-node rather than WASM, which matters on a box with one or two vCPUs. It's local: no API key, no network at inference time. The only network touch ever is the first init, which downloads about 90MB of quantized weights to a local cache. After that a warm embed runs in around 73ms.
One invariant is load-bearing enough to state out loud: fastembed L2-normalizes its output, so L2-distance ranking equals cosine ranking, and the vec0 tables can use the default L2 metric and stay correct. Swap in an embedder that doesn't normalize and the KNN order corrupts silently, with no error and no crash. That is the kind of thing you want written down before someone changes the model.
Vectors as a derived index
Here is the idea the whole design rests on. The vector rows are a derived index. They stand to the main tables the way an FTS index stands to a text column: a queryable overlay computed from data that is already correct on its own. They are not a store of record.
Follow that through and the operational properties fall out for free. If the extension fails to load, the process still boots, FTS5 still works, and the system is still correct; the only thing missing is semantic search. Nothing cascades. If the vec rows are dropped entirely, you rebuild them, because the turns, memory items, entities, and digests they were computed from all still sit in the main tables. Losing the vectors costs recall, not correctness.
That is the whole argument for keeping everything in one file: a store of record has to be durable and kept in sync, which is the job a separate vector service does, and a derived index needs neither, because you can recompute it from the main tables at will. Remove the second service and you lose nothing of record.
The 384 dimensions are fixed at migration time, and that is a real lock-in: changing the embedding model later means a new migration and a full re-embed of every row. But that is a bounded operation you schedule, not data you can lose.
Two mechanisms keep the derived index honest. First, prune-at-supersede: when writeMemory supersedes a memory item, it deletes the old row's vector in the same block that writes the new one. Skip it and superseded vectors pile up forever, crowding the top-k for any semantically similar query. (The original design pruned entity vectors at graph compaction; review caught that memory vectors needed the same treatment.) Second, the backfill script, which is the recoverability claim made executable: it is idempotent, its skip-set is the source ids already present in the matching vec table, it takes --dry-run, and it embeds with bounded concurrency, five at a time by default. It reports per-store counts and exits non-zero if any store errors.
When the vector arm fails
Because the index is derived, letting it fail is a design choice, not an accident to guard against. The degrade contract is uniform across all three hybrid paths. In mode:'hybrid', if the vector arm throws or returns nothing, you get the FTS arm's result. Semantic recall degrades to keyword recall. The one way hybrid hands back an empty list is if the FTS arm itself throws, and that gets a log line. In mode:'vector', a failure returns [], logged, no exception thrown, because there is no keyword arm to fall back on. An empty query gets FTS behavior under hybrid and [] under vector, with no embed call at all.
When both arms return, hybrid merges them with Reciprocal Rank Fusion. Each result scores by its rank in each list:
score = 1 / (60 + rank)
The two lists' scores are summed per source id, the merged list is sorted high to low, de-duplicated, and cut to limit. That is lib/rrf.ts: 36 lines, pure, no imports, no side effects. The constant 60 is the standard RRF damping term; it keeps any single list from winning on a rank-1 hit alone.
Two details keep the retrieval honest. The mode parameter defaults to "fts", so every existing caller behaves exactly as it did before; three MCP tools gain the option, chat_search, entity_search, and memory_recent. And every vector path over-fetches k = min(limit * 4, 256) rows before its post-KNN filters run, because active and conflict flags and id filters can drop hits after the KNN, and you don't want to hand back three results when the caller asked for ten.
Adding async without touching the callers
The vector path has one genuinely awkward requirement: it has to await embed(), which makes it async. But searchTurns already had a pile of synchronous callers, the extraction path among them, and turning them all async would have rippled through the codebase for no benefit to any of them. This is async contagion, and both usual outcomes are bad: either every call site changes, or the vector path can't be async at all.
TypeScript function overloads resolve it. The FTS signature stays synchronous; the vector one is async:
searchTurns(a, b, c): Turn[] // fts, unchanged
searchTurns(a, b, c, mode: 'vector' | 'hybrid'): Promise<Turn[]>Existing callers pass three arguments and keep getting a synchronous array; they never see the async overload. Only the three MCP dispatch cases pass the fourth argument and opt into the promise. memory_recent took a slightly different route, adding separate async functions and leaving the original synchronous ones untouched, but the principle holds: the new capability layers on top of the old surface instead of rewriting it.
A shape the type system can't see
The overloads are the type system doing its job. vecKnn() is where its guarantees stop. It declared its return type as { id: number | string; distance }[], but the query selected the raw column by name. For the conversation table that meant:
SELECT turn_id, distance FROM conversation_vec ... -- rows come back as { turn_id, distance }better-sqlite3 keys result rows by the actual selected column name, so the rows would have come back as { turn_id, distance } and h.id would have been undefined (the property just does not exist). npm run typecheck stayed green over the broken version; the declared type was a lie the compiler had no way to check. Review caught it before the code ever ran: every caller read h.id, and the vector arm would have passed [undefined, ...] into a SQL IN clause, which better-sqlite3 rejects at runtime. The core capability, a vector query finding an FTS miss and a hybrid merge on top of it, would simply not have worked. The fix is one clause, SELECT idColForTable(table) AS id, distance, aliasing the column back to the name the type promised. The point is narrow. Runtime shape divergence from a driver's column naming is where the type system's guarantees end.
What's running and what's still pending
The deploy landed on 2026-07-05. The code is in production, the vec tables exist, and semantic search is available at the tool layer. instrumentation.ts fires an embed('warmup') at service start, so the ~90MB model download happens at boot rather than on the first user turn.
Three gates remain before hybrid mode goes live, and they are mine to run, not the pipeline's. Confirm the warm-up line in the service logs. Run the backfill over SSH, dry-run first to check the counts, then for real to embed the historical rows. Measure free RAM after the model loads, alongside the Next.js process. If the headroom comes in under about 300MB, I set EMBED_INLINE=false before Rheo starts using hybrid. That flag moves conversation embedding off the logTurn write path and onto session close, where upsertDigest sweeps the session's unembedded turns; memory items, entities, and digests are low-volume and embed inline regardless. The trigger is a budget. If logTurn gains more than 300ms at p95 in production, inline embedding is the first suspect.
Some things were left out on purpose. memory_context, the single call that assembles the session-boot bundle, stays FTS-only in v1; to get semantic recall there, Rheo has to call chat_search or entity_search with mode:'hybrid' directly. There is no OpenRouter-hosted fallback embedder, deferred until the local model proves not good enough. There is no automatic backfill cron; it is an operator script. And there is no graph expansion over entity edges yet, which is Track 6, waiting on the edge data. None of those are failures; they are the scope line for Track 5. The next thing that happens here is the backfill run, once the warm-up line shows up in the logs.