NemoClaw Knowledge Wiki

Difference between RAG and Agents for workflow

7 min read

https://www.youtube.com/watch?v=WYqhc802nqk

Here is a detailed breakdown of the video “RAG vs Agents” by Dr. Anil Variyar.

Video Summary

Dr. Anil Variyar provides a visual and conceptual breakdown of the differences between Retrieval-Augmented Generation (RAG) and AI Agents, using a weather forecasting application as a running example.

  1. Large Language Model (LLM): The video starts by showing the limitation of a standard LLM. When asked for a future weather forecast, it cannot provide real-time, factual data because its knowledge is limited to its training data. It can only generate text based on learned patterns, which can lead to generic or outdated responses.
  2. Retrieval-Augmented Generation (RAG): RAG is presented as a solution to the LLM’s limitation. In a RAG system, a user’s query is first used to retrieve relevant, factual information from an external knowledge source (like a database or document store). This retrieved context is then provided to the LLM along with the original query, allowing the LLM to generate a more accurate and grounded response. The key characteristic of RAG is that it follows a pre-defined workflow set by the developer.
  3. AI Agents: Agents represent the next level of sophistication. An Agent is an LLM that is given a goal (via a system prompt), a set of available tools (e.g., APIs, databases), and the autonomy to decide how to use them. Instead of following a rigid, developer-defined workflow, the Agent plans and decides the sequence of actions on its own. It reasons about which tools to call and in what order to best accomplish its task, making it a more dynamic and powerful system.

The core distinction is that RAG and other workflows have their logic and data flow determined by the developer, whereas an Agent has the LLM itself determine the logic and data flow to achieve a given objective.

Detailed Video Breakdown

Here is a timestamped log of the video’s content, including visual descriptions and the speaker’s key points. 00:00 - RAG vs Agents: Introduction

  • Visuals: The title slide shows “RAG vs Agents”. On the left is a diagram where a query, “What will the weather be like in Mumbai this week?”, goes to an LLM. The LLM uses a “Weather Tool” to access a “Weather DB” and produces the answer: “Be prepared for warm temperatures and the possibility of rain.”
  • Speaker: Dr. Anil Variyar introduces the topic: a visual exploration of the difference between Retrieval Augmented Generation (RAG) and Agents.

00:13 - The Basic LLM

  • Visuals: A slide titled “LLM” appears. A user query “What is the weather in Mumbai like next week?” points to an icon representing an LLM (a neural network).
  • Speaker: The speaker starts with the fundamental component, the LLM. He uses the weather query as an example to be carried through the video. He clarifies that this is a conceptual app using an LLM API, not just typing into a public tool like ChatGPT.

00:55 - How an LLM Responds

  • Visuals: The text “LLM can only responds based on its neural network weights” is added to the slide. An arrow points from the LLM back towards where the answer would be.
  • Speaker: He explains that an LLM is a neural network whose parameters (weights) are set by its training data. Its primary function is to predict the next words to form a response based on these weights.

01:32 - LLM Limitations: Real-time Data

  • Visuals: The LLM’s generated response appears in a yellow box: “I’m sorry, but I can’t provide real-time weather updates. For the latest weather information in Mumbai, I recommend checking a reliable weather website or app, such as the Weather Channel, AccuWeather, or your local meteorological service.”
  • Speaker: Since the LLM does not have access to real-time information or forecasts, it provides a well-trained but unhelpful response, acknowledging its inability to answer factually.

02:03 - LLM with General Knowledge

  • Visuals: The query is changed to “What is the weather generally like in Mumbai?“. The LLM’s response now says: “Mumbai experiences a tropical climate, characterized by three main seasons: a humid tropical climate, a hot and dry season, and a monsoon season.”
  • Speaker: When asked a more general question, the LLM can answer based on the historical data it was trained on. However, this is still just a “prediction” based on its weights and doesn’t solve the problem of needing factual, up-to-date information.

03:16 - RAG (Retrieval-Augmented Generation)

  • Visuals: A new slide titled “RAG”. A query “Tell me about Nvidia’s revenue” is shown. This becomes a “Vectorized query” sent to a “Document Store” (a vector database). The store returns “Relevant Chunks” of information. Both the “Query” and “Relevant Chunks” are fed into the LLM, which then generates a factual answer: “Nvidia’s revenue is USD…”
  • Speaker: He introduces RAG as the solution. He explains the typical RAG workflow for document-based research: vectorize a query, retrieve relevant document chunks from a vector store, and feed both the query and the retrieved context to the LLM to generate a grounded answer.

05:20 - Applying RAG to the Weather Example

  • Visuals: The slide returns to the weather example, now showing a RAG workflow. The text “LLM Supplemented by Retrieved Information” is displayed. The query “What is the Weather in Mumbai like next week” goes to an LLM. This LLM extracts “location, dates”. This information is used to query a “Weather DB”. The DB returns structured data (the “Retrieval”). This data is sent to another LLM instance, which generates a summarized, natural-language forecast.
  • Speaker: He applies the RAG concept to the weather problem. An LLM first extracts entities (location, date) from the query. This information is used to retrieve data from a weather database. The retrieved data is then used to augment a final prompt to the LLM, which summarizes the information for the user.

07:07 - LLM Workflows

  • Visuals: A slide titled “WorkFlow” shows a more complex, predefined sequence. The query goes to an LLM for entity extraction, then to a “Weather DB”, then to a “Weather Tool” (e.g., for further computation), and finally to another LLM to generate the final response. The caption reads: “Work flow set by developer”.
  • Speaker: RAG is an example of a broader concept: an LLM Workflow. In a workflow, the developer explicitly defines the sequence of steps and tool calls. The LLM is used as a component (e.g., for processing or summarization) within this fixed structure.

08:31 - Agents

  • Visuals: A slide titled “Agent” is shown. A “System Prompt” box appears, stating: “You are a Weather Forecasting Assistant. Answer weather related queries using the tools provided”. The user query is sent to the LLM, which now has access to two tools: a “Weather Tool” and a “Weather DB”. The caption changes to: “Agent plans and decides workflow”.
  • Speaker: He introduces the concept of an Agent. The key difference is that instead of the developer defining the workflow, the Agent is given a high-level goal (via a system prompt) and access to tools. The LLM within the agent is responsible for planning the steps—deciding which tools to call, in what order, and how to use their outputs to answer the query.

10:41 - The Agent’s Final Output

  • Visuals: The Agent diagram is shown completing its task. It has autonomously used the tools to gather data and now generates a comprehensive answer: “This week, the weather in Mumbai is expected to be as follows…” followed by a detailed, structured forecast.
  • Speaker: He explains that the LLM in the agent has now planned and executed the tool calls to generate the final, detailed answer. The fundamental difference between a workflow and an agent is that the agent autonomously decides the workflow, rather than having it predefined by the developer. He concludes by stating that modern tools like ChatGPT are evolving to be a mix of both predefined workflows and more dynamic, agentic systems.

Graph View