Pathway’s BDH: Challenging Transformer AI with Continuous Learning and Latent Thinking
Clip title: The new architecture challenging AI’s dominant idea Author / channel: The Deep View URL: https://www.youtube.com/watch?v=fjB6sEPC4CE&t=747s
Summary
Zuzanna Stamirowska, CEO and co-founder of Pathway, discusses the limitations of current Large Language Models (LLMs) and introduces her company’s “post-Transformer” architecture designed to overcome these challenges. She explains that existing LLMs, largely based on the Transformer architecture (stemming from the “Attention is All You Need” paper), suffer from fundamental issues such as ephemeral context windows, a lack of continuous learning or long-term “parametric memory,” and “catastrophic forgetting.” This means that current models cannot truly learn from ongoing interactions or integrate new information into their core knowledge base; instead, they rely on storing contextual data akin to sticky notes or tattoos, necessitating repetitive and computationally expensive retraining for updates. Furthermore, their reasoning is often constrained by verbalized language, unlike the more abstract thought processes of humans.
Pathway addresses these shortcomings with its novel architecture, named BDH (Beautiful Dragon Hatchling), first published in October 2025 (as mentioned in the video). BDH aims to provide native memory, enabling continuous learning and evolution with experience, akin to how humans learn and internalize knowledge over time. Crucially, it supports “latent thinking,” which allows the model to reason in an abstract space, free from the limitations of verbalizing every step. This approach is intended to mimic human intuition and problem-solving, making the AI more adept at tasks like strategy games where non-verbal reasoning is paramount.
The development of BDH is rooted in a deep, theoretical understanding of complex systems, drawing on fields like theoretical computer science, economics, and physics, rather than purely empirical methods. This foundational approach allows Pathway to predict behavior across different scales and improve AI’s interpretability and safety, mitigating risks such as unintended consequences (like the “paperclip maximizer” problem). The architecture also boasts significant computational efficiency, allowing for the training of larger models with considerably less compute and at lower costs, addressing the sustainability challenge of current energy-intensive LLMs. Pathway is actively working with partners like AWS to integrate these models into real-world applications, focusing on use cases that demand long-horizon reasoning.
Video Description & Links
Description
What comes after large language models?
In this episode of The Deep View Conversations, we talked with Zuzanna Stamirowska, CEO of Pathway, to explore why her team believes today’s dominant AI architecture has fundamental limits, and what it could take to move beyond them.
Pathway is developing Dragon Hatchling, a new architecture designed to give AI native memory, continual learning, and a different approach to reasoning. Stamirowska explains why today’s LLMs can appear to remember without actually internalizing what they learn, why reasoning through language creates its own constraints and costs, and how Pathway is trying to build models that can think in a more abstract way.
The conversation looks at how those architectural changes could affect hallucinations, interpretability, safety, and the enormous compute demands of modern AI. Stamirowska shares how her background in complex systems and game theory shaped Pathway’s approach, why the company made an early bet on challenging the transformer, and how the AI coding revolution has already radically changed the way her own team works.
Topics covered: • Why transformers struggle with memory and continual learning • How Pathway’s Dragon Hatchling architecture works • How a different architecture could reduce compute costs • How interpretability could make advanced AI more predictable • Why Pathway’s engineers have largely stopped writing code themselves • How Stamirowska uses Codex, Claude Code, and other AI tools • Why leaders should be ruthless about identifying the critical path
If you’re interested in what could come after today’s LLMs, and whether the next big leap in AI will require more than simply scaling transformers, this conversation offers a fascinating look at one of the teams betting on a fundamentally different path.
TIMELINE 0:00 Introduction 1:58 What Pathway does 4:02 Why Pathway bet beyond the Transformer 13:00 The fundamental limits of LLMs 18:24 Why AI memory is an imitation 19:39 Reasoning beyond language 21:54 What is BDH? 23:19 Why “Dragon Hatchling”? 27:40 When people can try Pathway’s new models 31:39 Hallucinations, safety and interpretability 37:19 The paperclip problem 38:55 Why post-Transformer AI could cost less 43:11 Zuzanna’s path from game theory to AI 46:08 The neuroscience insight behind BDH 51:00 Why Pathway’s engineers stopped coding
Tags
ai, artificial intelligence, LLMs, transformers, pathway, Zuzanna Stamirowska, Jason Hiner, the deep view
Related Concepts
- Transformer architecture — Wikipedia
- Continuous Learning
- Latent Thinking
- Post-Transformer AI
- Large Language Models — Wikipedia
- Attention Mechanism — Wikipedia
- Catastrophic Forgetting — Wikipedia
- Complex Systems Theory — Wikipedia
- Game Theory — Wikipedia
- Computational Efficiency
- Paperclip Maximizer — Wikipedia
Related Entities
- Pathway
- Zuzanna Stamirowska
- The Deep View
- Gemini 2.5 Flash
- AWS — Wikipedia
- Attention is All You Need — Wikipedia