4-bit quantization: Technique enabling LLM training at 4-bit precision to drastically reduce costs. Example: Gemini Ultra (2023) training cost ~191M,[[entities/gpt−4∣GPT−4]](2023)78M (Altman claims higher). Recent 2025 advances make 4-bit training feasible for efficient model development. 4-bit quantization | 2026 04 14 How does 4bit quantisation work
Source Notes
2026-04-23: https://www.youtube.com/watch?v=-cRedoYETzQ Julia Turc The video discusses the evolution and challenges of training large language models (LLMs) with reduced precision, particularly focusing on the shift towards 4-bit floating-point (FP4) training. Cost of Training LLMs: Tr (How does 4bit quantisation work)