NemoClaw Knowledge Wiki

Karpathys LLM Wiki Beyond RAG for Persistent Knowledge Bases

4 min read

Karpathy’s LLM Wiki: Beyond RAG for Persistent Knowledge Bases

Clip title: Karpathy’s LLM Wiki: Watch Me Build a Knowledge Base From Scratch! Author / channel: Onchain AI Garage URL: https://www.youtube.com/watch?v=zVEb19AwkqM

Summary

The video introduces Andrej Karpathy’s innovative “LLM Wiki pattern,” a paradigm shift from traditional Retrieval Augmented Generation (RAG) models for managing personal knowledge bases. Karpathy’s approach advocates for a persistent, compounding wiki maintained entirely by a Large Language Model (LLM). The fundamental problem addressed is that traditional RAG systems repeatedly “rediscover” knowledge from scratch with each query, lacking accumulation, persistence of cross-references, and often locking user data within provider systems.

The core idea behind the LLM Wiki pattern is to offload the repetitive, painstaking work of knowledge maintenance to the LLM, allowing the human user to focus on higher-level tasks like curating sources and formulating insightful questions. The architecture comprises three layers: “Raw Sources” (immutable, read-only articles, papers, images, etc.), “The Wiki” (LLM-owned markdown files containing summaries, entities, and concepts, which the LLM creates, updates, and cross-links), and a “Schema” (a configuration file guiding the LLM on conventions and workflows, co-evolved by the user and LLM). The presenter uses the analogy of the wiki as a codebase, Obsidian (a popular markdown editor) as the Integrated Development Environment (IDE), the LLM as the programmer, and the schema as the style guide.

The system operates through three primary workflows: Ingest, Query, and Lint. The Ingest workflow involves feeding new source documents into a raw directory. The LLM automatically reads these, extracts key information, writes summary pages, updates existing concept and entity pages with integrated knowledge, flags contradictions, updates the master index, and logs all changes. This process incrementally “compiles” knowledge, making the wiki smarter with each new input. The Query workflow allows users to ask complex questions against the entire wiki. The LLM searches relevant pages, synthesizes answers, and, crucially, can file valuable answers back into the wiki as new, cross-referenced pages, leading to a compounding knowledge effect. Finally, Lint acts as a health-check mechanism, enabling the LLM to identify and suggest fixes for inconsistencies, missing links, or data gaps, thus ensuring the wiki’s integrity as it grows. The division of labor is clear: humans direct and think, while the LLM agent handles summarization, cross-referencing, and maintenance, overcoming the human tendency to abandon wikis due to high upkeep.

This LLM-driven wiki approach offers several philosophical advantages. It is explicit, providing a visible and navigable knowledge artifact without hidden embeddings or opaque memory; yours, as all data resides locally on your machine, free from vendor lock-in; built on file over app principles, utilizing universal markdown formats for interoperability with any tool; and supports a “bring your own AI” philosophy, allowing users to plug in various LLMs (like Claude, GPT, Codex, or open-source models), and even fine-tune them directly on their wiki’s data. This pattern is versatile, with potential applications ranging from in-depth research and personal self-improvement tracking to business intelligence, developing trading strategies, enhancing reading comprehension, and aiding due diligence.

The video concludes with a practical demonstration of building a “Trading Strategies Knowledge Base” using Claude Code. This involves setting up the directory structure, ingesting raw trading video transcripts, defining a schema for concepts and entities, and then showcasing how the LLM generates linked concept pages (e.g., “draw on liquidity,” “breaker blocks”). The demo vividly illustrates how querying the wiki can not only retrieve existing knowledge but also prompt the LLM to perform web searches for external information and integrate it, further enriching the knowledge base. The key takeaway is the creation of a persistent, ever-growing knowledge artifact where intelligence compounds over time, significantly reducing manual maintenance burden and empowering users with a highly customized, explicit, and extensible knowledge system.

Graph View