Compounding Knowledge
Compounding Knowledge refers to the approach of building persistent, evolving knowledge bases using large language models (LLMs) rather than relying solely on retrieval-augmented generation (RAG) systems. This methodology emphasizes the accumulation and refinement of structured information over time, allowing knowledge to build upon itself incrementally. The concept draws an analogy to compound interest in finance, where gains accumulate and reinvest to produce exponential growth.
Core Approach
Rather than treating each LLM interaction as a discrete query-response cycle, compounding knowledge systems maintain editable, updatable knowledge repositories that persist between interactions. As the system encounters new information or refines existing understanding, these changes are integrated back into the knowledge base itself. This creates a feedback loop where the quality and comprehensiveness of the knowledge base improve over successive interactions, with each refinement becoming available for future use.
Implementation and Examples
Andrej Karpathy’s LLM Wiki project exemplifies this approach, demonstrating how structured knowledge can be systematically built and updated within an LLM-augmented workflow. Such systems typically involve humans and LLMs collaboratively maintaining a knowledge base—identifying gaps, correcting errors, and organizing information in ways that become increasingly useful over time. This differs from RAG systems, which retrieve static external information to ground LLM responses without necessarily improving the underlying knowledge source.
The advantage of compounding knowledge lies in its potential to create more reliable and specialized knowledge resources tailored to specific domains or use cases. As the knowledge base grows more comprehensive and accurate, subsequent queries can draw on higher-quality information, potentially reducing hallucinations and improving the relevance of LLM outputs.
Source Notes
- 2026-04-07: Karpathy
- 2026-04-10: Karpathys LLM Wiki Beyond RAG for Persistent Knowledge Bases · ▶ source
- 2026-04-12: Heres what it actually does how to build it yourself
- 2026-04-19: Karpathy Loop Auto Optimize AI Inhuman Iteration for Agent Improvement · ▶ source
- 2026-04-27: AI Context Layer Architectures: Karpathy