NemoClaw Knowledge Wiki

MedGemma 27B - Fahd Merza

4 min read

https://www.youtube.com/watch?v=QBuBvMA0oSw

The video provides a comprehensive overview and demonstration of Google’s new MedGemma 27 billion parameter model, highlighting its capabilities in medical text and image comprehension. Model Overview and Capabilities: MedGemma is a medical AI model developed by Google, built on the Gemma 3 architecture. It is specifically trained for medical text and image comprehension tasks and comes in three variants: a 4 billion multimodal model (available in pre-trained and instruction-tuned versions) and a 27 billion model available in both text and multimodality versions. All MedGemma variants utilize a SigLIP image encoder, pre-trained on de-identified medical data including chest X-rays, dermatology images, ophthalmology images, and histopathology slides. Their language components are trained on diverse medical datasets, including medical text, question-answer pairs, and electronic health records (FHIR-based). Demonstrations:

  1. Simulated Pre-visit Intake Demo: The video first showcases MedGemma acting as an AI agent to gather patient information for a pre-visit report. It dynamically updates a report with details like primary concern, history of present illness, relevant medical history, and medications, as the conversation with a simulated patient progresses. The patient persona in the demo is “Sacha Silva,” a 24-year-old female with asthma, presenting with flu symptoms.
  2. Local Installation and Text Inference: The speaker demonstrates installing MedGemma locally on an Ubuntu system with an NVIDIA H100 PCIe GPU (requiring 48GB of VRAM). The installation involves creating a Conda virtual environment, installing necessary prerequisites like PyTorch and Transformers, and logging into Hugging Face to access the gated model. For text inference, MedGemma acts as a helpful medical assistant. When asked to differentiate between bacterial and viral pneumonia, the model provides a detailed and grounded response covering causative agents, onset, and progression.
  3. Image Inference - X-ray Analysis: The video shows MedGemma analyzing an AI-generated X-ray image. The model is prompted to act as an “expert radiologist” and describe the X-ray. It provides a comprehensive analysis, including image description (standard PA chest X-ray, skeletal structures, gray scale), key findings and interpretation (bones, lungs, heart, soft tissues), and an overall impression (normal-appearing chest X-ray).
  4. Image Inference - Ophthalmology: MedGemma is tasked with analyzing an AI-generated ophthalmology image. As an “expert ophthalmologist,” it analyzes the image, identifies it as “highly stylized, possibly artistic or abstract,” and notes that it’s “not a clinical photograph or a standard anatomical diagram.” It then breaks down potential “problems” or “observations” regarding iris structure, vascularization, sclera/cornea, and overall structure, while maintaining that it’s not a realistic depiction.
  5. Image Inference - Dermatology: Finally, the model is tested on a dermatoscopic image of a pigmented lesion. Acting as an “expert dermatologist,” MedGemma analyzes the image, providing an image analysis, systematic evaluation based on ABCDE criteria (Asymmetry, Border irregularity, Color variation, Diameter, Evolution), dermoscopic features (pigment network, globules, streaks, dots, regression areas, vascular patterns), and a differential diagnosis (ranking possibilities from melanoma to benign nevus).

Disclaimer and Conclusion: The video strongly emphasizes that MedGemma models are for educational purposes and should not be used for self-diagnosis or as an alternative to human medical professionals. Its primary use is to empower medical practitioners and improve healthcare quality. The speaker concludes by praising MedGemma as a “very, very impressive model” that can be used in various AI-powered healthcare applications due to its reliability in critical domains.

Graph View