Gemini Ultra

Training Cost: The 2023 training cost for Google’s Gemini Ultra was estimated at $191 mi ll i o n (p er a [[e n t i t i es / s t an f or d ∣ St an f or d]] re p or t), s i g ni f i c an tl y hi g h er t han [[e n t i t i es / g pt - 4∣ GPT - 4]]^{'} s$ 78 million (though Sam Altman claimed it was even more).
4bit Quantisation: The video How does 4bit quantisation work (by Julia Turc) discusses the evolution of training LLMs with reduced precision, particularly the shift towards 4-bit floating-point (FP4) training as a technique to reduce computational and memory costs in large model training.

2026 04 14 How does 4bit quantisation work

Source Notes

2026-04-23: https://www.youtube.com/watch?v=-cRedoYETzQ Julia Turc The video discusses the evolution and challenges of training large language models (LLMs) with reduced precision, particularly focusing on the shift towards 4-bit floating-point (FP4) training. Cost of Training LLMs: Tr (How does 4bit quantisation work)