DeepSeek-OCR: Artificial Forgetting for Efficient AI Long Text Compression
Clip title: Why DeepSeek Wants AI To Forget Author / channel: Two Minute Papers URL: https://www.youtube.com/watch?v=4fLVq6HJsKc
Summary
This video introduces DeepSeek-OCR, a novel approach to compressing long text documents for Artificial Intelligence (AI) models. The core problem addressed is that traditional AI models process text as “tokens,” and every word read incurs a cost, making extensive reading or long conversational contexts prohibitively expensive. Analogizing human memory, which tends to forget perfect details over time but retains essential information, DeepSeek-OCR proposes “artificial forgetting” for AI to reduce this burden.
DeepSeek-OCR achieves this by converting text documents into images, then employing a vision encoder to compress these images into a significantly smaller number of “vision tokens.” This process is likened to JPEG compression for images, but it’s a learned, token-based compression specifically optimized for text content. Remarkably, the system can achieve a 10x compression ratio while retaining over 96% of the original text’s precision. Even at nearly 20x compression, it maintains approximately 60% accuracy, demonstrating an impressive ability to discard redundant information without losing critical context.
The technical ingenuity lies in its tokenizer, which is designed to handle very high-resolution documents efficiently. It utilizes a combination of local window attention, a 16x convolutional compressor, and global visual processing, allowing it to process vast amounts of data without an explosion in computational cost. This isn’t merely about efficiently storing information; the true power of DeepSeek-OCR lies in its capacity to generate massive amounts of training data for Large Language Models (LLMs) and Visual Language Models (VLMs). It can produce over 200,000 pages of training data per day on a single A100 GPU, significantly accelerating the development and training of new AI models.
However, the video also highlights an important limitation: OCR alone is insufficient to fully validate true context optical compression. When visual tokens are severely limited, the model may start to “fill in the gaps” by guessing missing or ambiguous text based on linguistic context and learned priors, rather than solely relying on visual evidence. This means that at extreme compression levels, the AI might act more like an advanced auto-complete system, inferring information rather than perfectly recalling it from the compressed visual data.
Despite these nuances, DeepSeek-OCR is presented as a groundbreaking, open-source system that promises to democratize AI development. By drastically reducing the computational and memory requirements for processing documents, it enables AI models to handle larger contexts more affordably and facilitates the creation of vast training datasets, fostering the development of new and improved open models across various domains like fluid simulation, neural rendering, and robot learning.
Video Description & Links
Description
📝 The DeepSeek OCR paper is available here: https://github.com/deepseek-ai/DeepSeek-OCR
Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi
Tags
ai, deepseek, deepseek ai
URLs
Related Concepts
- Artificial Forgetting
- Long Text Compression
- Memory Efficiency
- Vision Encoder
- Open-Source AI
- Large Language Models — Wikipedia