CMA-ES Decoding Optimization for MedGemma: A Flat Landscape on Chest X-Ray Reports

Vision-language models (VLMs) for medical imaging rely on decoding parameters, mainly the sampling temperature and the nucleus sampling (top-p) threshold. These are typically set by convention. We apply the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search jointly over temperature and top-p for MedGemma 4B-IT on chest X-ray report generation using PadChest-GR. Our fitness function is a Wasserstein-distance proxy in sentence-embedding space between generated reports and the expert reference. Across five independent runs (749 evaluations), the landscape is flat, varying by only 2.6-4.5% within each run, and no CMA-ES configuration improves over the published default (greedy decoding) on held-out test data. The baseline achieves near-maximal semantic consistency (0.97), which CMA-ES reduces to 0.82. Both configurations detect fewer than 5% of ground-truth clinical findings, and CMA-ES raises the rate of “normal” templates on confirmed-pathology samples from 37% to 66%. For this zero-shot model the landscape is flat under an embedding-space proxy; whether this holds under entity-level clinical metrics (e.g., RadGraph F1) is open. Under our metrics, the bottleneck is visual perception, not text generation. Code: https://github.com/michaelcw02/ cmaes-medgemma-xray.