NemoClaw Knowledge Wiki

Unsloth Deep Research vs. Web Search: Performance and Quality Analysis

4 min read

🗂️ AI & Agents · View mindmap

Unsloth Deep Research vs. Web Search: Performance and Quality Analysis

Clip title: Unsloth’s Local Deep Research: Is it any good? Author / channel: Learn Meta-Analysis URL: https://www.youtube.com/watch?v=ZdUJDbXF8v4

Summary

The video provides a comparative analysis of Unsloth’s “Deep Research” feature against its “Web Search” functionality, aiming to determine which is more effective for different research needs. The presenter, Noah, used a practical, non-academic query (“I want to know all the common issues with the 2022-2026 Ford 6.7L turbo diesel engine”) across various large language models (LLMs) including OSS 120b, Qwen 3.6 35b, Nemotron 3.5, Trinity Mini 20b, Granite 4 Small, and LFM 2.5 2.6b. The tests were performed on a system with an RTX 4060 (8GB), 64GB DDR5, and an i5 processor, using maximum context where possible.

Key findings regarding efficiency revealed that Unsloth’s Web Search consistently outperformed Deep Research in terms of speed. For instance, Deep Research with OSS 120b took nearly 30 minutes, whereas Web Search completed the task in about 13.5 minutes (46% of the time) while conducting multiple searches. The difference was even more pronounced with Qwen 3.6, where Deep Research took 27 minutes compared to just under 5 minutes for Web Search (18%). Although Trinity Mini 20b’s Web Search was remarkably fast at 47 seconds (7.5% of Deep Research time), it only performed a single search. The smallest model, LFM 2.5, notably failed to generate a report in Deep Research mode, though its Web Search still provided multiple results.

In terms of output quality and presentation, Deep Research offered a more structured approach, presenting a detailed plan that could be edited, citing sources specifically within the text, and aiming for better synthesis of information in a comprehensive report format. Conversely, Web Search provided a more direct, chat-like response with descriptive listings. Noah found that the OSS 120b’s Web Search output was effective for his initial query, offering a complete list of common issues in an organized table without prioritizing them, which suited his specific need for comprehensive information. He also appreciated Qwen’s structured output for its clear presentation of issues, symptoms, causes, model years, and fixes.

A critical observation arose from an academic research query which specifically requested peer-reviewed sources on Noah Schroeder’s papers concerning LLMs in education. While Deep Research (OSS 120b) failed to produce a complete sentence, Web Search (Qwen 3.6) launched numerous tool calls and gathered about 120 sources. It successfully identified 4 out of 5 of Noah’s publications and accurately summarized his published position. However, it hallucinated that Noah was an author on a paper he did not write, which is a significant flaw concerning academic integrity. This test highlighted the challenge of distinguishing reliable sources from “AI slop websites” that both features tended to reference.

In conclusion, Noah suggests that for most general research tasks, Unsloth’s Web Search is the superior option due to its significantly faster execution and sufficiently good quality of results. He finds the extended time taken by Deep Research not always justified by the marginal improvements in synthesis or presentation for casual use. For academic research, both features fall short. The hallucination issue in Web Search and the general inability to constrain searches to highly specific and reputable sources (like academic databases only) in either feature make them unreliable for rigorous scholarly work at present. Despite these limitations, Noah continues to use Unsloth Studio, primarily relying on its Web Search capabilities through the UI.

Video Description & Links

Description

I was excited about seeing local deep research in Unsloth Desktop. I tested deep research and web search across 6 different models. I was surprised by the results.

Relevant Links for this video: My results: https://docs.google.com/document/d/1nmGFOYE2ABqjv_VZPhn7Nb4O9VogUIgGSC4FB55bW18/edit?usp=sharing Unsloth Deep Research PR on github: https://github.com/unslothai/unsloth/pull/7219

Meta-Analysis and Systematic Review Tutorials: Meta-analysis in JASP: https://www.youtube.com/playlist?list=PLXa5cTEormkFfajjr1tWBzR6EsVtzpxPp Conventional meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkEbYpBIgikgE0y9QR7QIgzs Three-level meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkHwRmu_TJXa7fSb6-WBXXoJ Three-level meta-analysis with correlated and hierarchical effects and robust variance estimation: https://www.youtube.com/playlist?list=PLXa5cTEormkEGenfcnp9X5dQUhmm7f9Jp

LLMs and PC Stuff: Linux for Academics: https://youtube.com/playlist?list=PLXa5cTEormkFnjfVW-tUQIQnHi87xsdEK&si=kkswGQXAxG7pVtkC

Full-text finder: https://youtu.be/7DkGD29xJB4

URLs

YouTube Playlist URLs

Graph View