title: “RAG”
RAG
Retrieval-Augmented Generation (RAG) is a framework used to optimize the output of a Large Language Model (LLM) by retrieving relevant, authoritative information from an external knowledge base to augment the model’s context window.
Paradigms & Evolutions
- Traditional RAG: Relies on the retrieval of discrete, often static, document chunks to ground model responses in external data.
- LLM Wiki pattern:
- Employs an LLM to autonomously maintain and evolve a structured wiki.
- Focuses on a self-sustaining, continuously updating knowledge architecture rather than reactive retrieval of isolated snippets.
- Reference: 2026 04 10 Karpathys LLM Wiki Beyond RAG for Persistent Knowledge Bases
- Summary:
- Introduced by andrej
- Domain-Specific Fine-Tuning: Uses linear adapters to optimize embedding models for specific domains without full retraining or re-embedding, enabling cost-effective performance improvements. (Reference: inbox
- Context Engineering: A new AI skill differentiating from prompt engineering, focusing on structured context provision for LLMs. Defined by Tobi Lütke (Shopify CEO) and Andrej Karpathy as “the art/science of providing all” [incomplete quote]. Emphasizes systematic context design over ad-hoc prompting. Reference: 2026 04 14 Context engineering by prompt engineering channel
Source Notes
- 2026-04-07: Structured AI Context Beyond RAG Limitations with Map First Architectu · ▶ source
- 2026-04-08: Chroma Context 1 Self Editing Search Agent for Efficient RAG · ▶ source
- 2026-04-10: LlamaIndex’s LiteParse: Agentic Document Processing and the End of Frameworks Clip title: LiteParse - The Local Document Parser Author / channel: LlamaIndexs LiteParse Agentic Document Processing and the End of)
- 2026-04-14: # RAG plus knowledge graphs using GRAPHITI --- --- https://www.youtube.com/watch?v=PxcOIINgiaA This video By Cole Medin provides a comprehensive overview of Retrieval Augmented Generation (RAG) and introduces Graphiti, an open-source platform designed to address RAG’s limitatio (RAG plus knowledge graphs using GRAPHITI)
- 2026-04-23: Engine Survival: The Critical Role of Oil Pressure and Warning Lights · ▶ source