NemoClaw Knowledge Wiki

Claude code - Yifan - Beyond the Hype channel

5 min read

https://www.youtube.com/watch?v=i0P56Pm1Q3U The video delves into the question of why Claude Code, despite using the same underlying LLM models as other coding agents, often feels significantly better to use. The speaker reverse-engineers Claude Code to uncover its “secret sauce,” concluding that it lies primarily in extensive and sophisticated prompt engineering. Reverse Engineering Process:

  1. Initial Attempt (Deobfuscation): The speaker initially tried to deobfuscate Claude Code’s bundled cli.js file (a massive 9MB, 443,063 lines of JavaScript) using webcrack. This revealed many dependencies but the core LLM prompts were dynamically constructed, making direct extraction difficult.
  2. Key Insight (Proxying API Calls): The breakthrough came from realizing Claude Code’s documentation allowed overriding the ANTHROPIC_BASE_URL environment variable. This indicated direct LLM API calls, which could be intercepted.
  3. Interception (Proxyman): Using Proxyman, the speaker successfully intercepted all requests between Claude Code and the Anthropic API. This revealed a “ton of messages” being sent, confirming that Claude Code was indeed orchestrating complex LLM interactions.

The Core LLM Agent Loop: The intercepted requests revealed the fundamental loop structure for Claude Code’s LLM interactions:

  • System Prompt: Defines the agent’s role and overarching rules (always included at the beginning of the message history).
  • Tool Definitions: Specifies the tools available to the LLM (e.g., file system access, Bash commands).
  • User Message: The user’s current input or query.
  • LLM API Call: Claude Code sends the complete message history (System Prompt + Tool Definitions + User/Assistant/Tool messages) to the Anthropic LLM.
  • LLM Response: The LLM responds with either an assistant message (text) or a tool call.
  • Tool Execution: If a tool call is made, Claude Code’s local runtime executes the specified tool.
  • Tool Result: The output of the tool execution is added back to the message history as a “Tool” message.
  • Iteration: The loop continues, with the updated message history, until the LLM provides a final, conclusive assistant message without further tool calls.

The speaker notes that much of Claude Code’s excellent user experience is built upon this fundamental local orchestration loop. Key Observations from the System Prompt (**system_prompt_main.md**):

  1. Repetition and Emphasis: Crucial workflows, important instructions, and key tool usages (e.g., the TodoWrite tool for task management, or reminders to run lint/typecheck) are reiterated multiple times across different sections of the system prompt. Directives like “IMPORTANT”, “VERY IMPORTANT”, “NEVER”, and “MUST” are frequently used to emphasize critical behaviors, significantly improving the accuracy of function calling and adherence to rules. The speaker humorously notes that agents “forget,” and this repetition serves as a consistent reminder.
  2. Workflows Defined in Natural Language: A significant portion of Claude Code’s operational logic, task management strategies, code style guidelines, and tool usage policies are meticulously described in natural language within the system prompt, rather than being hardcoded in JavaScript. This design makes the agent highly adaptable; simply modifying the prompt can alter its core behavior.
  3. Importance of Formatting: The prompt heavily utilizes markdown formatting (headings, bullet points, code blocks) and XML-like tags (e.g., <example>, <system-reminder>). These aren’t just for human readability; they add semantic meaning that the LLM interprets, helping it understand and respect the prompt’s structure and intent. For instance, <system-reminder> blocks are actively inserted into the message history by Claude Code after every task progression and tool call to reinforce key directives to the LLM.

Sub-Agents Feature: Claude Code’s recently introduced sub-agents feature allows the main agent to delegate complex, specialized tasks to smaller, dedicated agents.

  • Delegation: The main agent makes a tool call to launch a sub-agent (e.g., “requirement-analyzer”). This tool call includes a detailed prompt outlining the sub-agent’s task, context, and desired output.
  • Isolated Memory: Crucially, when a sub-agent is triggered, it operates with its own, independent message history. All of the sub-agent’s internal LLM interactions (its own assistant messages, tool calls, and tool results) are discarded once its task is complete.
  • Summarized Return: Only the sub-agent’s final, summarized assistant message is returned to the main agent, and it’s treated by the main agent as a tool call result. This prevents memory bloat in the main agent’s context window and keeps its focus on the higher-level task.
  • Prompt-Driven Sub-agent Invocation: The ability for the main agent to launch and manage sub-agents is itself defined as a detailed tool description within the main system prompt. This description specifies when to use different sub-agents, how to handle their outputs, and even emphasizes that sub-agent invocations are stateless.

General Principles for Building Effective LLM Agents: From this analysis, key principles emerge for building capable LLM agents:

  1. Sophisticated Prompt Engineering: Invest heavily in crafting detailed, clear, and well-structured prompts.
  2. Repetition and Emphasis: Don’t shy away from reiterating crucial instructions and desired behaviors within the prompt, using emphasis to highlight importance.
  3. Natural Language Workflow Definition: Define agents’ core workflows and decision-making processes primarily through natural language in their system prompts, reducing reliance on hardcoded logic.
  4. Semantic Formatting: Utilize markdown and XML-like tags to add structure and semantic meaning to your prompts, helping the LLM better interpret and follow instructions.
  5. Modular Sub-agents: Employ sub-agents for complex tasks, ensuring they operate with isolated memory and return only summarized results to the main agent. This is vital for managing context window limitations and maintaining overall agent focus.
  6. Model-Specific Prompt Tuning: Recognize that prompt tuning is highly specific to the LLM model family being used. Continuous evaluation (evals) and iterative refinement of prompts are essential to optimize performance for a given model.

The video concludes by emphasizing that prompt engineering remains a critical skill in the evolving AI landscape, and that applying these principles can empower developers to build their own highly effective coding agents.

Graph View