Karpathy's LLM Wiki: The Knowledge Base That Beats RAG

Andrej Karpathy published the pattern on April 4, 2026 as a public GitHub gist (the LLM Wiki gist). It's framed not as a finished product but as an "idea file" — a sketch of how an LLM can incrementally compile curated source material into a persistent, interlinked Markdown wiki.

The architecture has three layers. Raw sources are immutable. The wiki sits on top -- a directory of markdown entity pages linked with [[wiki-link]] syntax. The schema is a config doc (Karpathy uses the CLAUDE.md convention) defining how the wiki behaves. Karpathy's own wiki reportedly reached ~100 articles and 400,000 words and was still faster and more accurate than his RAG pipeline.

Karpathy describes exactly three operations against the wiki, and they are remarkably symmetrical. There is no chunking strategy, no embedding model selection, no hybrid retrieval, no re-ranker. The "retrieval" is the LLM reading index.md and following links.

Vector-store RAG retrieves raw fragments per query -- the fragments are dumb, do not know about each other, and cannot flag contradictions. The wiki inverts the cost model: knowledge is compiled once on ingest, then read cheaply forever. When the LLM answers a question it is reading three or four pre-synthesised entity pages, not a hundred raw fragments.

Agent Vault is the production system we run for Jordan James Media's client portfolio. About 200 entity pages across 12 projects, mirrored to a Supabase table, with an ingest-on-correction hook that auto-promotes user pushback into a new entity page. The pattern maps almost perfectly to Karpathy's three layers -- memory files are entity pages, MEMORY.md is the index, and CLAUDE.md is the schema.

Three pieces still missing: formal [[wiki-link]] syntax across all files (we use raw paths and grep), a multi-page ingest workflow that updates 10-15 pages per source (our ingest is mostly single-page right now), and one consolidated log.md (each session writes its own log, which is more useful for our case but loses the global grep view).

What we did just ship after writing the first draft of this post: the memory-lint skill, in three flavors. Structural lint (regex-driven) runs at every session start and on demand — it caught four real drift bugs in our 442-memory corpus on its first run, all fixed the same day. Semantic lint (Claude Haiku 4.5) runs weekly with a --since 7d filter so the cost stays around ten cents a week — it reads each memory plus the files it cites in backticks, flags specific claims that look stale or contradicted. A write-time hook (PostToolUse) lints every memory edit in real time, so new bad [[wiki-link]] references can't silently enter the corpus.

Here is the cleanest way to frame the difference. RAG is a search engine for your documents -- it answers "what did source X say about Y." The LLM Wiki is an encyclopedia your AI writes for itself -- it answers "what do I currently believe about Y, given everything I have read." Those are different questions, and the second one is almost always what you actually want.

Start small. One markdown directory. One CLAUDE.md. One index.md. One log.md. Let your LLM read it, write to it, and lint it. After six months you will have something a new agent can pick up in five minutes -- which is the actual test of whether your knowledge base works.