# Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology

Source: [Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology](<https://www.sota2.com/research/paper/6a34cd57206542251d7c61fa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-24T05:00:13.022Z

Authors: Yusuf Salcan, Simon Ging, Robin Tibor Schirrmeister, Philipp Arnold, Elmar Kotter, Behzad Bozorgtabar, Thomas Brox (1) \_\_INSTITUTION\_7\_\_ Computer Vision Group, University of Freiburg, Germany, (2) Adaptive & Agentic AI \_\_INSTITUTION\_9\_\_ Lab, Aarhus University, Denmark, (3) Department of Radiology, Medical Center -- University of Freiburg, Germany, (4) CRIION-AI Lab, Freiburg, Germany)

Published: 2026-06-18T16:55:26.000Z

## Abstract

We study how to train visually grounded vision-language models (VLMs) for radiology without manual spatial annotations. We introduce RefRad2D, a large-scale bilingual (German/English) dataset of 1.2M CT and MR image-text pairs derived from clinical practice, with task-specific VQA and spatial grounding subsets generated automatically via LLM-based curation and automated segmentation. Trained on this data, our model RadGrounder jointly performs report generation, visual question answering, and spatial grounding via bounding-box detection or segmentation. On external VQA benchmarks (Slake, VQA-RAD), RadGrounder achieves competitive results with specialized medical VLMs. Adding our clinical data to the training mixture improves open-ended VQA over fine-tuning on the downstream datasets alone, showing the transferability of our dataset. Crucially, adding grounding supervision does not degrade language quality, enabling spatially verifiable outputs at no cost to VQA performance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Medical Visual Question Answering on SLAKE 12](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-slake-12.md>): Medical Visual Question Answering; Slake 12; F1: 88.4.

- [Medical Visual Question Answering on VQA-RAD 10](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-vqa-rad-10.md>): Medical Visual Question Answering; VQA-RAD 10; Closed Accuracy: 68.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.20477>)

[PDF](<https://arxiv.org/pdf/2606.20477.pdf>)