Cross-references

Cross-referencing in a wiki context involves linking related content within the same or different documents to enhance navigation and understanding. It’s particularly useful for maintaining coherent, interconnected knowledge bases.

Karpathy’s LLM Wiki Pattern

• Introduced by Andrej Karpathy as an alternative to traditional Retrieval Augmented Generation (RAG) models.
• Emphasizes a persistent, compounding wiki maintained entirely by a Large Language Model (LLM).
• Addresses the challenge of effectively managing and growing personal knowledge bases over time.

Key Concepts

• Retrieval Augmented Generation (RAG): A technique that combines vector-based document retrieval with language generation.
• Persistent Knowledge Base: A system that accumulates information continuously, allowing for the gradual refinement and expansion of data.
• Large Language Model (LLM): Advanced AI models trained on vast amounts of text data to generate human-like responses.

Advantages

• Dynamic updating without manual intervention.
• Continuous compounding of knowledge through interactions with users.
• Enhanced accuracy and depth as the model learns from its own outputs.

Integration

The video Karpathy’s LLM Wiki: Watch Me Build a Knowledge Base From Scratch! highlights how this approach can be implemented to create an effective, self-maintaining personal wiki.

2026 04 10 Karpathys LLM Wiki Beyond RAG for Persistent Knowledge Bases