Memory strategies¶

The rule: turns is source of truth; everything else is a projection that can be dropped and rebuilt. This page maps strategies fitting that rule, says which ships today, and names the seams alternatives plug into.

Forward-looking work lives in Roadmap. Implementation details for what ships live in Context pipeline.

The four families¶

Tiered virtual memory with self-edited blocks (Letta / MemGPT). LLM mutates its own context via tools. Rejected — destructive; see Decisions.
Bi-temporal append-only knowledge graphs (Zep / Graphiti, Cognee). t_valid / t_invalid per edge; new facts mark old ones invalid; nothing deleted. Adopted incrementally — bi-temporal facts shipped (M1; see Bi-temporal facts), M5 plans Graphiti as a swappable projector.
Self-evolving atomic notes with emergent linking (A-MEM, generative-agents memory stream). One observation per row, LLM-generated metadata, links decided at write-time. Familiar-Connect's facts and people_dossiers are this family today. M2 (importance) and M3 (citation-bearing reflections) close the gaps.
Hand-authored, rule-activated character context (RisuAI / SillyTavern lorebooks). Keyword-activated entries. character.md ships as the always-on degenerate case (persona plus operational essentials); M4 added a keyword-activated lorebook alongside it — see Lorebook.

What's wired today¶

Side-index	Writer	Read by
`fts/turns/` (tantivy)	sync write from `append_turn`	`RagContextLayer`
`fts/facts/` (tantivy)	sync write from `append_fact`	`RagContextLayer`
`summaries`	`SummaryWorker`	`ConversationSummaryLayer`
`facts`	`FactExtractor`[^claims]	`RagContextLayer` (via FTS)
`facts.superseded_*`	`FactSupersedeWorker`	every fact reader (filters retired)
`people_dossiers`	`PeopleDossierWorker`	`PeopleDossierLayer`
`reflections`	`ReflectionWorker`	`ReflectionLayer`
`fact_embeddings`	`FactEmbeddingWorker` (M6, opt-in)	`RagContextLayer` (rerank)

Every writer is watermark-driven and idempotent; deleting a side-index table rebuilds it from turns. See Context pipeline for watermark semantics.

append_fact suppresses near-duplicates at insert: when a new fact's normalized text (lowercased, whitespace-collapsed, surrounding quotes/punctuation stripped) matches an existing current (non-superseded, non-expired) fact for the same familiar_id and the same subject-key set, the insert is skipped and the existing fact returned — no new row, no supersede. This kills alias/nickname restatement pile-up ("Cor is called Cor", once per turn batch) that bloated dossiers and RAG. Comparison is deterministic exact-match after normalization; semantic/embedding-based dedup is a deliberate non-goal here (would risk false-positive suppression of genuinely distinct facts) — a possible follow-up via fact_embeddings.

Tantivy commits on Windows occasionally race with antivirus segment-scans (PermissionDenied on a fresh .term file). FtsIndex._commit retries those transient locks with short backoff; if the commit still can't land, HistoryStore logs a warning and keeps going so one FTS write can't tear down the bot. The SQL row remains canonical and rebuild_fts will pick up any skipped doc.

Reads are multi-signal: BM25, recent-window exclusion, a 1-10 importance hint per fact (M2), and cosine similarity to the cue embedding (M6, opt-in).

Swap points¶

1. Layer order and selection (`Layer` Protocol)¶

Per-channel prompt_layers in character.toml swaps order or disables layers per Discord channel. New layer = one class implementing build(ctx) + invalidation_key(ctx), registered in commands/run.py::_default_assembler.

M3 (ReflectionLayer) and M4 (LorebookLayer) plug in this way. A candidate replacement layer can also run side-by-side: register both, switch via TOML on a test channel.

2. Projection writer (`MemoryProjector` Protocol)¶

The watermark-driven writers — SummaryWorker, FactExtractor, PeopleDossierWorker, ReflectionWorker, FactSupersedeWorker — implement a thin :class:MemoryProjector Protocol (M5):

class MemoryProjector(Protocol):
    name: str
    async def run(self) -> None: ...

Names registered today (built-ins):

Name	Class	Side-index
`rolling_summary`	`SummaryWorker`	`summaries`
`rich_note`	`FactExtractor`	`facts`
`people_dossier`	`PeopleDossierWorker`	`people_dossiers`
`reflection`	`ReflectionWorker`	`reflections`
`fact_supersede`	`FactSupersedeWorker`	`facts.superseded_at` / `superseded_by`
`fact_embedding`	`FactEmbeddingWorker`	`fact_embeddings`

Operators pick the active set in character.toml:

[providers.memory]
projectors = ["rolling_summary", "rich_note", "people_dossier", "reflection", "fact_supersede"]

Default lists all five built-ins (fact_embedding is registered but opt-in — the seam is wired but stays off until the operator picks an embedder backend). Drop a name to disable that writer. Add a third-party projector (Graphiti, Cognee, …) by calling register_projector("graphiti", factory) at import time and listing it here. Unknown names fail loudly at config load — a typo never silently drops a writer. Empty list disables all projection (side-indices stop refreshing; reads still work against whatever exists).

[^claims]: Extraction prompt enforces claim/event discipline: one speaker's assertions about another person stored attributed ("X claims …"), never as flat fact; roleplay and running bits recorded as bits, not real events. Identity ties to the Participant canonical_key, never a name adopted in play: impersonation bits ("No I am Cor") never mint identity facts and never merge two people; world trivia a speaker mentions isn't a fact about them. Guards against memory poisoning via in-character narration — extractor sees every turn whether or not the familiar engaged. Activity-return turns are skipped wholesale, keyed on the turns mode column ("activity_return"); the [returned from <label>] prefix stays as a display marker only. Experience text is self-generated fiction, and the activity is already recorded as a mechanical event-fact — see Activities.

3. Retrieval ranking (`RagContextLayer`)¶

Four signals fuse at rank time, TOML-driven:

[memory.retrieval]
bm25_weight       = 1.0
recency_weight    = 0.0
importance_weight = 0.6   # M2 — fact's 1-10 hint
embedding_weight  = 0.0   # M6 — needs an embedder + populated index

The layer over-fetches BM25 candidates (up to 4× max_rag_facts), normalises each signal to [0, 1] within the candidate batch, and keeps the top N by weighted sum:

BM25 quality — best score in the batch maps to 1.0.
Recency — newest fact id in the batch = 1.0.
Importance — importance/10. Legacy / unscored rows (importance IS NULL) get the neutral midpoint, never zero.
Embedding — cosine similarity to the cue, mapped from [-1, 1] to [0, 1] via (cos + 1) / 2. Candidates lacking a stored vector get the neutral midpoint (0.5) — same shape as importance. The cue embedding is computed only when at least one candidate has a stored vector, so a cold side-index pays no embed cost.

importance_weight = 0 reproduces pre-M2 ordering; embedding_weight = 0 (default) reproduces M2-era ordering. See Tuning — retrieval ranking.

Embeddings (M6)¶

Vector side-index over facts for paraphrase-tolerant recall, fused into the existing BM25 + importance rerank.

fact_embeddings carries (fact_id, model, dim, vector, created_at) keyed on (fact_id, model). Vectors are packed little-endian float32 in a BLOB column — no sqlite-vec dependency yet; brute cosine over BM25-overfetched candidates suffices at per-host volumes. Pairing the row key with model (the embedder's Embedder.name) means a backend swap accumulates new rows beside the old; reads filter on the active model so prior runs stay queryable for audit but don't leak into ranking.

FactEmbeddingWorker is the projector: every 15 s, pulls up to 32 current facts (superseded_at IS NULL) lacking a row for the active model, runs the embedder, persists each vector. The watermark is implicit — the LEFT JOIN fact_embeddings filter is the watermark. A model swap, a deleted side-index, or an extension of the active embedder all converge on the same idempotent backfill.

RagContextLayer reads vectors at rerank time. When embedding_weight > 0 and the layer has an Embedder, it pulls candidate vectors with one batched query, embeds the cue once, and fuses cosine similarity. Missing vectors map to the neutral 0.5 so an unembedded candidate isn't penalised relative to a 5/10 — same legacy-friendly shape as importance.

The seam splits across three knobs so each can move independently:

[providers.embedding].backend — picks the backend. off (default), hash (no-deps baseline), or fastembed (production-grade ONNX sentence embedder; needs the local-embed extra).
[providers.memory].projectors — list "fact_embedding" to start populating the side-index.
[memory.retrieval].embedding_weight — turn the signal on at read time.

To enable, flip all three; to disable, drop embedding_weight back to 0 (reads quiet) or drop fact_embedding from the projector list (writes quiet). The side-index can be deleted anytime — the next worker tick rebuilds it from facts.

FastEmbedEmbedder carries the model identifier in its name (fastembed:BAAI/bge-small-en-v1.5); since the storage row key is (fact_id, model), a model upgrade accumulates new vectors beside the old. Old rows stay queryable for audit but don't leak into active rank — reads filter on the live embedder's name. To reclaim space, delete superseded rows after switching:

sqlite3 data/familiars/<id>/history.db \\
    "DELETE FROM fact_embeddings WHERE model != 'fastembed:BAAI/bge-base-en-v1.5';"

Lorebook (M4)¶

Hand-authored, keyword-activated canon. Lives at data/familiars/<id>/lorebook.toml:

[[entries]]
keys      = ["paris", "france"]
content   = "Paris is the capital of France. Founded ~250 BCE…"
priority  = 100
selective = false       # any key matches; default

[[entries]]
keys      = ["dragon", "wyvern"]
content   = "Dragons in this world breathe radiant light, not fire."
priority  = 50

LorebookLayer reads the file on every assemble (cheap; cached by the Assembler's invalidation key), scans the active channel's last recent_window turns case-insensitively against each entry's keys, and renders the matching subset newest-priority-first under a ## Lorebook block. selective = true flips per-entry match from any-key (OR) to all-keys (AND) — useful when a generic key ("dragon") shouldn't fire alone without a disambiguator ("dragon" + "Cassidy").

No worker; the file is the sole source of truth. Operators edit it in place, no migration. The cache key combines a content hash of the file with matched entry indices, so the layer only flips when the file or activation set actually changes.

Relevant knobs live in [budget.<tier>]:

[budget.text]
lorebook_tokens      = 800   # cap on the rendered block
max_lorebook_entries = 10    # hard cap on rendered entries

Authored canon stays separate from experiential memory (see below) — the lorebook never mutates on its own.

Reflections (M3)¶

Higher-order syntheses over recent turns + facts. Each reflection row carries:

text — one or two sentences naming a pattern, recurring tension, open question, or theme.
cited_turn_ids / cited_fact_ids — forever-provenance JSON arrays. Citations render as breadcrumbs [T#42, F#7] so the reading model can map a synthesis back to its source.
last_turn_id / last_fact_id — snapshot of the worker's view at write time. Citations render against this snapshot; the worker's watermark lives in a separate reflection_watermark table.

ReflectionWorker ticks every 60 s; fires when at least turns_threshold (default 20) new turns have arrived since the last watermark. It builds a prompt over the new turns (capped at max_turns_per_tick, default 50, taking the most-recent tail) plus the 20 most recent facts, asks for at most 3 reflections per tick, and persists each answer citing at least one valid turn or fact id. Uncited answers are dropped — a free-floating opinion isn't a synthesis.

The watermark advances to latest_turn at the end of every tick, even when the LLM returns [] or every item is filtered. Otherwise a no-substance reply would pin the worker to a growing unprocessed window, and the next tick would re-send the same (plus-more) turns — the prompt would balloon into 100k+ tokens every 60 s. Capping max_turns_per_tick is the second guardrail: even a fresh worker against a long-lived database can't ship hundreds of turns at the LLM in one tick.

ReflectionLayer reads the most recent rows scoped to the active channel (channel-agnostic rows surface in every channel), renders citation breadcrumbs, and flags (stale) on rows citing at least one superseded fact. Stale rows are never deleted; the audit trail is the point.

Relevant knobs live in [budget.<tier>]:

[budget.text]
reflection_tokens     = 800   # cap on the rendered block
max_reflections       = 5     # hard cap on rendered rows

Bi-temporal facts¶

Each row in facts carries two independent time axes:

System-time (created_at, superseded_at, superseded_by) — when we recorded it, and when we retired it. Supersession keeps the old row; the new row points back via superseded_by.
World-time (valid_from, valid_to) — when the fact was observed to apply in the world. valid_from defaults to the source turn's timestamp; the LLM may override with an explicit ISO-8601 string on spotting an "as of …" phrase ("Aria moved to Berlin in early 2024"). valid_to is NULL while the fact still applies.

Default reads stay "current truth": superseded_at IS NULL and valid_to IS NULL OR valid_to > now. Audit queries pass as_of=<datetime> to recent_facts / search_facts / facts_for_subject; that switches to a bi-temporal slice (valid_from <= as_of < valid_to) and includes superseded rows so prior beliefs remain reconstructable.

Legacy rows (pre-M1) carry valid_from = valid_to = NULL and read as "always valid"; no backfill — the feature is forward-only.

Different writers populate the two axes; do not conflate them:

valid_to is for world-time only — set when the speaker explicitly anchors the end of a fact in real time ("until last June", "ended in 2019"). The FactExtractor prompt steers the LLM away from setting it just because a fact looks outdated or has been replaced.
superseded_at / superseded_by are for retirement bookkeeping — set by FactSupersedeWorker, which evaluates each newly-arrived fact against prior current facts about the same subject and asks the background LLM which (if any) to retire, via HistoryStore.supersede() (existing-id form). A prior already retired by a concurrent writer is recorded in the result's skipped rather than rewiring the chain twice. Superseding also deletes the retired fact's subjects' people_dossiers rows (same familiar_id): the dossier worker compounds prior text and never un-bakes a retired fact, so a surviving row would keep the stale prose alive. Dropping the row makes the worker's next tick rebuild it clean from current facts (the prior=None path). The watermark is deliberately not reset instead — a kept row re-compounds the poisoned text via its "Previous dossier" prompt. Subject-less facts have no dossier to drop.

A fact can be retired (system-time) while its world-time validity stays open: e.g., the extractor re-renders the same belief with sharper wording — the older row is superseded but its valid_to stays NULL. Conversely, a fact whose valid_to lies in the past is not superseded; it just no longer applies in the world. Read helpers filter on both columns, so either condition hides a row from "current truth" reads.

Why rich-note + bi-temporal, not graph-only¶

Live disagreement in the field: knowledge graph (Zep, Cognee, GraphRAG) vs richly-attributed flat notes with emergent links (A-MEM, generative agents). Graphs win on multi-hop relational queries, lose on token cost. Rich-note systems are simpler, cheaper, and degrade on multi-hop.

For a Discord familiar, multi-hop queries are rare ("what's Alice's drink?", "what were we talking about yesterday?" dominate). Rich-note + bi-temporal handles them at lower cost. Graphiti-style graph stays on the roadmap (M5) as an additional projector, not a replacement.

Why authored canon stays separate from experiential memory¶

Authored canon (character.md, lorebook.toml) — human-edited, never evolves on its own.
Experiential memory (turns and projections) — bot-generated, evolves every interaction.

Mixing them lets the agent rewrite its own character description. Splitting them keeps the trust boundary inspectable.

Operator playbook¶

Rebuild a side-index¶

sqlite3 data/familiars/<id>/history.db "DELETE FROM facts;"
# next FactExtractor tick rebuilds from turns

Force a dossier re-fold¶

UPDATE people_dossiers
SET dossier_text = '...', last_fact_id = 0
WHERE canonical_key = 'discord:1234';

last_fact_id = 0 forces the worker to refold every prior fact on its next tick.

A/B a layer on one channel¶

[channels.<id>].prompt_layers overrides default layer order on one Discord channel. Compare a test channel (with the candidate layer) against a control. Once A1 lands, the same mechanism extends to STT, turn detection, and voice pipeline mode.