Can Gemini transcribe a song?
Clip title: Can Gemini transcribe a song? Author / channel: Google Cloud Tech URL: https://www.youtube.com/watch?v=UnkUpqNv-10
Summary
Here’s a summary of the video:
In the video, Gabe Weiss, a Data Cloud Developer Relations Manager at Google Cloud, discusses his excitement for Gemini 4 and its potential to assist musicians. As a musician who plays the mandolin, Gabe highlights the common challenge of finding sheet music for obscure or newly discovered songs. He challenges Gemini 4 to generate sheet music for an old folk tune called “La Basura.”
The video shows Mollie Pettit testing different versions of Gemini to generate the sheet music. First, they try Gemini 3.5 Flash Lite, which Gabe evaluates while playing the mandolin. He notes that the generated sheet music is in the wrong key and has incorrect clefs, resulting in a tune that is far from the original song.
Next, they try Gemini 3.8 Flash, which shows some improvement with the correct key, but the notes and time signature are still off. Finally, they try Gemini 4 Argon, which gets much closer to the original tune, though still with some errors in time signature and notes. Gabe is impressed by how Gemini 4 Argon gets close enough for him to pick up the rest by ear, demonstrating the potential of AI in assisting with complex musical tasks like solving the polyphonic problem.
Video Description & Links
Description
Gemini 4 Argon → https://g.dev/ai/gemini-about Google Gemini models → https://g.dev/ai/gemini-models-docs Watch the full episode → https://g.dev/youtube/gemini-models-full
In this video, Luke Schlangen and Mollie Pettit team up with fellow Googler Gabe Weiss on a classically difficult problem in audio machine learning: transcribing a multi-instrument, polyphonic audio track into playable sheet music.
Watch as they throw it at three different Gemini models (Gemini 3.5 Flash-Lite, Gemini 3.8 Flash, and the new Gemini 4 Argon) and listen to Gabe play the results on the mandolin.
Chapters: 0:00 – Gabe’s challenge: Transcribing tunes from audio to mandolin sheet music 1:30 – Gemini Flash-Lite: Wrong key, wrong clefs, and definitely not the song 2:10 – Gemini Flash: Right key, closer structure, but reversed sections 3:19 – Gemini 4 (Argon): It’s pretty close. 4:18 – Hearing the actual tune: How close did Gemini 4 really get? 5:20 – Cracking the “Polyphonic Problem”: Why this is classically hard for AI
More resources: Learn more about Gemini 3.8 Flash → https://goo.gle/3TuVfmT Builders → https://g.dev/ai/builder-member-awards
Watch more The Agent Factory → https://www.youtube.com/playlist?list=PLIivdWyY5sqLXR1eSkiM5bE6pFlXC-OSs
GoogleCloud Gemini GeminiAtWork Gemini4Argon GeminiFlash AIAgents GenerativeAI
Speakers: Luke Schlangen, Mollie Pettit, Gabe Weiss Products Mentioned: Gemini, Gemini 4 Argon, Gemini 3.8 Flash, Gemini 3.5 Flash-Lite, Google Antigravity
URLs
- https://g.dev/ai/gemini-about
- https://g.dev/ai/gemini-models-docs
- https://g.dev/youtube/gemini-models-full
- https://goo.gle/3TuVfmT
- https://g.dev/ai/builder-member-awards
- https://www.youtube.com/playlist?list=PLIivdWyY5sqLXR1eSkiM5bE6pFlXC-OSs