Reduced precision
Use of lower-precision data types (e.g., 8-bit, 4-bit) instead of standard 32/64-bit floating-point to reduce computational/memory costs in machine learning systems.
- 4-bit training evolution: Enables direct training of large language models (LLMs) at 4-bit floating-point (FP4) precision, reducing memory bandwidth and computational requirements compared to traditional 16/32-bit training 4-bit
- Cost reduction: Training costs for state-of-the-art LLMs remain extremely high (e.g., Gemini Ultra training cost ~78M; Sam Altman claims higher) LLM training costs
- Key application: 4-bit quantization addresses scalability challenges in [[concepts/large-language-model
- Reasoning efficiency: ThinkingCap-Qwen3.6-27B: Evaluating LLM Reasoning Efficiency and Accuracy demonstrates that fine-tuned models like Qwen3.6-27B]] can achieve the same accuracy as base models while reducing “thinking” steps by 36%, offering a new dimension of computational cost reduction beyond parameter quantization.