Engram
deepseek’s Engram is a technique for large-language-models that introduces a new axis of sparsity through conditional memory via scalable lookup.
Core Problem
Current Transformer architectures waste computation by failing to distinguish between:
- Tasks requiring deep thought (computationally intensive)
- Tasks requiring simple recall (which could be handled via memory lookup)
Key Innovation
Engram implements a scalable lookup mechanism to conditionally access memory for simple recall tasks, reducing unnecessary computation while maintaining performance on complex reasoning tasks.
2026 04 14 DeepSAeek Engram paper Prompt Engineering channel