Karpathy Stack · architecture v2

An engineering blueprint
for an LLM knowledge base

From messy pasted notes to an auditable, correctable, reusable second brain. This is the system diagram I drew after Karpathy's talk.

KEY INSIGHTThe original Karpathy setup misses one piece:
a feedback loop so errors flow back and get fixed.
STEP 01
Capture
Web clips, PDFs, podcast transcripts, chat logs
STEP 02
Denoise
Strip nav bars, ads, repeated passages, low-signal material
STEP 03 · CORE
Wikify
Structure into linked notes with entities, relations and properties together
STEP 04
Use
Agents retrieve, answer and write back at any time
Chapter One

Why notes
aren't enough

When your knowledge outgrows your memory,
you don't need more files, you need a navigable map.

Problem · Solution

Original vs upgraded

Original Karpathy

Write once

Capture → transcribe → archive; wrong stays wrong. No loop, no correction, and more notes means more mess.

Upgrade v2

Feedback loop

When AI uses the knowledge base it logs every miss, hallucination and stale fact, then feeds them back into the source files so notes correct themselves.

Regular node
Core node · feedback loop entry
— Data flow    ┈┈ Feedback loop
Implementation · Skill Manifest

How to build the feedback loop

A 100-line Agent Skill that writes how well the AI worked back into the vault as revision records.

# skills/wiki-feedback/SKILL.md
name: wiki-feedback
trigger: "after every retrieval"

on_hit:     record(query, path, used=true)
on_miss:    record(query, reason="not-in-vault")
on_wrong:   record(query, correction, path)

nightly:
  - aggregate misses → suggest new notes
  - aggregate wrongs → diff-patch old notes
  - commit to git, open PR for human review
System Diagram

The full feedback loop

Sources web · pdf · chat Clean + Split defuddle / chunker Vault (Wiki) markdown · links bases · canvas Agent Use retrieve / answer FEEDBACK · wrong / miss NIGHTLY · suggest new sources
After 6 Months

Data from six months of v2

13

Key optimisations · all shipped

-62%

Hallucination rate (vs. no feedback loop)

+4.1×

Single-query retrieval hit rate

227 auto revisions

189 were approved and merged by a human; 38 were rejected (data archived).

412 new notes

Clustered from miss logs, each with a traceable source.

Next Step

Start your Wiki v2

No need to rewrite every note. Wire in one loop and every AI use improves your vault.

TONIGHT
Install the Skill
pnpm i -g @lewis/wiki-feedback
DAY 2
Run for 7 days
Watch the miss log build up on its own
DAY 8 · CORE
First PR review
Spend 15 minutes reviewing the auto-generated revisions
MONTH 1
Start trusting it
Your vault comes alive
END · blueprint v2

Thanks · thanks

Blueprints, Skill and note templates are all at github.com/lewis/karpathy-wiki-v2