Pathway's BDH Architecture: Advancing AI Beyond Current LLM Limitations

Pathway’s BDH Architecture: Advancing AI Beyond Current LLM Limitations

Clip title: The new architecture challenging AI’s dominant idea Author / channel: The Deep View URL: https://www.youtube.com/watch?v=fjB6sEPC4CE

Summary

The video features an insightful conversation with Zuzanna Stamirowska, CEO and co-founder of Pathway, discussing the future of AI beyond current Large Language Models (LLMs). The main topic revolves around Pathway’s pioneering efforts to develop a new AI technology and architecture, dubbed BDH (Beautiful Dragon Hatchling), designed to overcome the inherent limitations of existing LLMs and foster the next generation of AI advancements. Pathway’s overarching vision is to build AI that learns autonomously, evolves continuously, and reasons with unprecedented efficiency and sustainability.

Zuzanna elaborates on the fundamental challenges associated with current LLMs, which are primarily based on the Transformer architecture. She highlights a significant “memory problem,” explaining that these models are trained once, rendering their knowledge static. This leads to issues like “catastrophic forgetting,” where new information can overwrite previously learned data, akin to a continuous Groundhog Day scenario. While external databases and context windows are used as workarounds, they don’t represent true, internalized learning. Another critical limitation is the “reasoning problem.” Current LLMs verbalize every step of their thought process in natural language, which is computationally intensive and restricts abstract reasoning. Unlike human chess players who intuit moves without articulating each step, LLMs struggle with abstract problem-solving, making them inefficient for certain complex tasks.

Pathway’s BDH architecture directly addresses these limitations. It integrates “native memory” into the learning process, allowing the AI to learn continuously from experience without forgetting past interactions or knowledge. Furthermore, BDH enables “latent thinking,” which is the ability to reason in abstract space, bypassing the need for explicit verbalization at every step. This abstract reasoning capability not only boosts performance in complex problem-solving but also dramatically improves computational efficiency and reduces energy consumption, making AI more sustainable. The architecture also enhances “interpretability and safety” by providing a clearer understanding of the AI’s internal workings, thus mitigating risks like hallucinations and facilitating more predictable and controllable behavior, avoiding scenarios like the “paperclip maximizer” problem.

Zuzanna’s diverse background in complex systems, game theory, theoretical computer science, economics, and physics provided a unique foundation for Pathway’s innovative approach. The company strategically committed in 2024 to building a differentiated AI architecture, focusing on foundational changes rather than incremental improvements to existing models. Pathway has already published its seminal paper and released a basic, open-source version of BDH, with plans for product launches in partnership with AWS and various design partners. The ultimate goal is to evolve the field of AI from an empirical discipline to a foundational science, leading to truly autonomous, continually learning, and reasoning systems with predictable and explainable behaviors.

Video Description & Links

Description

What comes after large language models?

In this episode of The Deep View Conversations, we talked with Zuzanna Stamirowska, CEO of Pathway, to explore why her team believes today’s dominant AI architecture has fundamental limits, and what it could take to move beyond them.

Pathway is developing Dragon Hatchling, a new architecture designed to give AI native memory, continual learning, and a different approach to reasoning. Stamirowska explains why today’s LLMs can appear to remember without actually internalizing what they learn, why reasoning through language creates its own constraints and costs, and how Pathway is trying to build models that can think in a more abstract way.

The conversation looks at how those architectural changes could affect hallucinations, interpretability, safety, and the enormous compute demands of modern AI. Stamirowska shares how her background in complex systems and game theory shaped Pathway’s approach, why the company made an early bet on challenging the transformer, and how the AI coding revolution has already radically changed the way her own team works.

Topics covered: • Why transformers struggle with memory and continual learning • How Pathway’s Dragon Hatchling architecture works • How a different architecture could reduce compute costs • How interpretability could make advanced AI more predictable • Why Pathway’s engineers have largely stopped writing code themselves • How Stamirowska uses Codex, Claude Code, and other AI tools • Why leaders should be ruthless about identifying the critical path

If you’re interested in what could come after today’s LLMs, and whether the next big leap in AI will require more than simply scaling transformers, this conversation offers a fascinating look at one of the teams betting on a fundamentally different path.

TIMELINE 0:00 Introduction 1:58 What Pathway does 4:02 Why Pathway bet beyond the Transformer 13:00 The fundamental limits of LLMs 18:24 Why AI memory is an imitation 19:39 Reasoning beyond language 21:54 What is BDH? 23:19 Why “Dragon Hatchling”? 27:40 When people can try Pathway’s new models 31:39 Hallucinations, safety and interpretability 37:19 The paperclip problem 38:55 Why post-Transformer AI could cost less 43:11 Zuzanna’s path from game theory to AI 46:08 The neuroscience insight behind BDH 51:00 Why Pathway’s engineers stopped coding

NemoClaw Knowledge Wiki

Explorer

Pathway's BDH Architecture: Advancing AI Beyond Current LLM Limitations