Yuwei Zhang

Decoding Claude Code's Memory System Design

A design read of the memory system behind Claude Code

on Apr 2026

Memory Storage: Markdown-Based Layered Persistence Over Vector Databases 🍅

Claude Code splits memory into two layers with a clean division of responsibility.

Explicit memory is written by humans, for Claude. It lives in CLAUDE.md or .claude/rules/ and carries project architecture overviews, coding conventions, and workflow constraints; in essence, an instruction set for how to work in this codebase. Its job is to encode team experience and rules so that Claude behaves like an onboarded teammate instead of defaulting to generic, web-average conventions.

Implicit memory is Claude’s own notes to itself. During each session the system observes the conversation and action trail; when trigger conditions fire, it calls a smaller model to distill the key information and write it to persistent files. The two layers form a dual track: explicit memory supplies constraints and direction from above, while implicit memory accumulates feedback from practice below.

The more interesting decision is the substrate. Where most AI memory systems reach for embeddings and vector retrieval, Claude Code persists all memory as Markdown files under ~/.claude/projects/<project>/memory/:

  • MEMORY.md is a lightweight index (≤200 lines / ≤25KB, loaded at startup) listing titles and short descriptions, enough for the model to know what exists without loading any of it.
  • Topic files (*.md) hold the actual content: structured YAML frontmatter (title, description, type, date, etc.) followed by free-form body text.

This choice trades retrieval infrastructure for legibility. Markdown files can be read, diffed, hand-edited, and deleted by the user with ordinary tools; there is no embedding index that can drift out of sync with the content it describes, and no separate database to operate. The index-plus-topic-file split is itself a form of progressive disclosure: pay a small fixed cost at startup for awareness, defer the full cost of any memory until it is actually needed.

Memories arrive by three routes (the main agent writing on user instruction, an Extract Memories pass after a session, or the user editing files by hand), and all routes converge on topic files in memdir, with MEMORY.md kept in sync for discoverability. The same model extends further without changing shape: team memories in a team/ subdirectory, isolated Agent Memory directories, and KAIROS append-only logs that get distilled back into topic files and the index.

Memory Retrieval: Intelligent Recall Powered by Sonnet 🍅

Retrieval is where the no-vector-database decision has to prove itself. On every user query, findRelevantMemories.ts uses Sonnet (a model call, not embedding similarity) to select at most five memory entries for injection into the current context.

The pipeline: recursively scan the .md files and extract frontmatter; filter out entries already shown this session; run a lightweight side-query (Sonnet, no main conversation history, max_tokens 256, JSON Schema output) to make the semantic selection; then read the selected files in full, appending a staleness warning to any entry older than one day.

The whole thing runs in parallel with the main API call, as a prefetch. If the prefetch loses the race, that round simply skips and retries on the next iteration; retrieval is never allowed to add latency to the main path. The selection policy is deliberately conservative, and the asymmetry justifies it: missing one relevant memory costs a little quality, while injecting a wrong one pollutes the context that every subsequent step reasons from. Prefetch aborts when its scope ends; skip conditions include single-word prompts, session byte limits, and empty optional memory lists.

graph LR
    Start((START)) --> Step1[1. Scan MD & Build List]
    Step1 --> Step2[2. Filter Displayed]
    Step2 --> SonnetAPI(3. Semantic Filtering)
    SonnetAPI --> Step4[4. Read File Content]
    Step4 --> CheckStaleness{5. Is Stale?}
    CheckStaleness -- "Yes - Over 1 Day" --> AddWarning[6. Append Warning]
    CheckStaleness -- "No" --> FinalStep[Ready to Inject]
    AddWarning --> Context((CONTEXT))
    FinalStep --> Context

Auto Dream: Post-Processing for Bloated Memories 🍅

Any memory store that only ever appends will degrade: entries go stale, near-duplicates accumulate, and the index drifts away from the files it describes. Auto Dream is the consolidation engine that pushes back. Gated, low-frequency triggers after sessions spin up a restricted forked agent that runs a four-phase pass over memdir: scan the structure, gather change signals, consolidate (merge duplicates, fix timestamps, delete conflicting entries, prune dead references), and update MEMORY.md.

Operating Loop: Extract → Store → Inject 🍅

Extract happens in-session: observe the dialogue, distill on trigger, write Markdown topic files, update MEMORY.md.

Consolidate runs offline via Auto Dream (/dream), local and lock-protected, with no external network calls.

Inject happens at session start: load the first 200 lines of MEMORY.md plus all CLAUDE.md tiers, priority low → high: org rules → project rules → user rules → CLAUDE.local.md. The ordering matters: the closer a rule sits to the individual developer, the later it loads and the more it can override.

Together the loop delivers three properties:

  • Automatic learning: the system summarizes what worked and what failed without manual curation.
  • Relevance-driven injection: only the most relevant slice enters context, so memory never crowds out the actual task.
  • Working continuity: technical decisions and project background carry across sessions instead of being re-derived each time.

Takeaways for the Agent Platform 🍅

Insight 1: Distill execution traces into reusable experience

When a Workflow shows stable success patterns, auto-distill them into implicit memory or reusable Skills in a structured Skill library. The trace is already paid for; the distillation step is what converts it from a log into an asset.

Insight 2: Weakly structured memory as a Skill precursor

Build a progressive Knowledge → Memory → Skill pipeline: lightweight Markdown memory entries first, promotion to high-reliability formal Skills only once an entry has proven itself. Requiring full structure up front kills capture; requiring none makes the store unusable, and a staged pipeline avoids both failure modes.

Insight 3: Offline memory housekeeping

Adopt Auto Dream’s locate → collect → consolidate → prune-and-index loop to keep context high-signal. Consolidation is cheap offline and expensive in-session, so scheduling it away from the interactive path is the right default.

Conclusion 🍅

Claude Code’s memory design shows what it looks like when agent memory graduates from a feature into infrastructure: a dual-track split between human-authored rules and self-distilled experience, layered Markdown persistence that stays legible and hand-editable, and offline consolidation that keeps the store trustworthy over time. For long-horizon collaborative agents, that combination is a pattern worth refering.