NemoClaw Knowledge Wiki

OpenRAG - IBM Channel

3 min read

OpenRAG - IBM Channel

https://www.youtube.com/watch?v=qreMmsOY86A Here is a summary of the video “What is OpenRAG?” featuring IBM’s David Jones-Gilardi, formatted as a Markdown document.

OpenRAG: Agentic RAG Systems Explained

Introduction

As Generative AI models mature, Context Windows have become significantly larger. However, even with the potential for “infinite” context windows, Retrieval-Augmented Generation (RAG) remains a critical architecture for modern AI systems. RAG is the method of injecting specific, external information into a model at runtime—information it wasn’t originally trained on, such as domain-specific knowledge or protected corporate data.

Why RAG Still Beats “Infinite” Context

Even if a model can take a million tokens, RAG is preferred for three main reasons:

  1. Cost: Most model providers charge by the token. Ingesting an entire library for every query is prohibitively expensive.
  2. Performance: Processing massive amounts of data in a single context window takes significantly more time (latency) than retrieving a targeted chunk.
  3. Accuracy: Models perform better and provide more precise responses when given the exact information they need rather than being forced to find a “needle in a haystack.”

The OpenRAG Stack

OpenRAG is an IBM-led open-source platform of tightly integrated tools designed to stand up an agentic RAG system in minutes. A complete system requires three core pillars:

1. Ingestion: Docling

  • The Problem: Documents like PDFs contain tables, images, and complex layouts that traditional text-parsers often mangle.
  • The Solution: Docling provides intelligent document ingestion. It identifies headers, tables, and images, extracting them into a format optimized for LLMs and AI agents. This ensures “clean” data enters the system.

2. Retrieval: OpenSearch

  • The Role: Functions as the “High-Speed Librarian” (Vector Database).
  • How it works: Once Docling processes a document, the data is converted into vector representations and stored in OpenSearch. This allows for lightning-fast similarity searches to find relevant context for a user’s query.

3. Orchestration: LangFlow

  • The Role: The “Wiring and Execution Engine.”
  • How it works: LangFlow ties everything together. It connects the data sources to the models (like Anthropic or Granite) and manages the logic of how an agent decides to search, retrieve, and answer.

How It Works

Once OpenRAG is installed, the workflow is straightforward:

  • Ingest Knowledge: Users can upload various document types (PDFs, docs) or provide URLs on the fly.
  • Chat with Data: Users can query their entire corpus of knowledge or use filters to search specific subsets of documents.
  • Customization via Studio: If you need to change your model provider or add an external data source, you can do so visually within the LangFlow Studio. Changes made in the flow are reflected in the OpenRAG UI immediately.
  • Extensibility: Developers can use the OpenRAG UI as a reference or leverage the LangFlow API to build completely custom applications on top of the stack.

Key Takeaway

OpenRAG is designed to move developers from zero to agentic search in minutes. It provides a pre-configured, production-ready stack that is fully open-source, allowing for deep manipulation of the AI pipeline without starting from scratch.

For more information, visit the IBM OpenRAG repository. https://github.com/langflow-ai/openrag

Graph View