Draft Model
A Draft Model (also known as a “fast model” or “proposal model”) is a smaller, computationally cheaper neural network used in conjunction with a larger Target Model to accelerate text generation. This architecture is central to Speculative Decoding, a technique that allows the system to generate multiple tokens in parallel using the draft model, which are then verified by the target model in a single forward pass.
Core Mechanism
- Proposal Phase: The draft model predicts a sequence of tokens based on the current context.
- Verification Phase: The target model evaluates the entire sequence of proposed tokens simultaneously.
- Acceptance/Rejection: Tokens are accepted if the target model’s probability distribution aligns with the draft’s predictions; otherwise, the sequence is rejected or partially accepted, and the target model generates the next token autoregressively.
Recent Developments & Implementations
- DeepSeek DSpark: A specialized inference acceleration framework introduced by DeepSeek that acts as a “speed layer” for Large Language Models (LLMs). It enhances speculative decoding to significantly reduce latency, reportedly doubling inference speed for models like Qwen3. See DeepSeek DSpark: LLM Inference Acceleration via Enhanced Speculative Decoding for detailed analysis.