# Medical Visual Question Answering on MMMU Med Source: [Medical Visual Question Answering on MMMU Med]() Collection index: [llms.txt]() Data updated: 2026-06-08T06:46:02.277Z Evaluation of model performance on the MMMU Med dataset for medical visual question answering, reporting scores across BMS, CM, DLM, P, PH, and Average metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Medical Visual Question Answering | | Dataset (including any stated split) | MMMU Med | | Metric | Average Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average Score](): Higher is better; unit: PERCENT. - [BMS Score](): Higher is better; unit: PERCENT. - [CM Score](): Higher is better; unit: PERCENT. - [DLM Score](): Higher is better; unit: PERCENT. - [P Score](): Higher is better; unit: PERCENT. - [PH Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average Score | Result date | Report paper | | --- | --- | --- | --- | | M3LLM-8B | 62.7 | Not recorded | 692d2110d6b0dd3342931689 | | Lingshu-7B | 60.7 | Not recorded | 6a264c8f90966ade919bea04 | | InternVL3-8B | 57.3 | Not recorded | 692d2110d6b0dd3342931689 | | QWen2.5-VL-7B | 54.7 | Not recorded | 692d2110d6b0dd3342931689 | | Lingshu-7B | 54 | Not recorded | 692d2110d6b0dd3342931689 | | MEDSIGHT | 51.3 | Not recorded | 6a264c8f90966ade919bea04 | | HuatuoGPT-Vision-7B | 50.7 | Not recorded | 692d2110d6b0dd3342931689 | | MedGemma-27B | 49.3 | Not recorded | 692d2110d6b0dd3342931689 | | HealthGPT-L14 | 49.2 | Not recorded | 6a264c8f90966ade919bea04 | | HealthGPT-14B | 48 | Not recorded | 692d2110d6b0dd3342931689 | | Qwen3-VL | 46.7 | Not recorded | 6a264c8f90966ade919bea04 | | InternVL3.5 | 46 | Not recorded | 6a264c8f90966ade919bea04 | | InternVL2 | 43.3 | Not recorded | 6a264c8f90966ade919bea04 | | HuatuoGPT-Vision | 43.3 | Not recorded | 6a264c8f90966ade919bea04 | | HealthGPT-M3 | 43.3 | Not recorded | 6a264c8f90966ade919bea04 | | Llama-3.2 | 39.3 | Not recorded | 6a264c8f90966ade919bea04 | | LLaVA-Med-7B | 38.7 | Not recorded | 692d2110d6b0dd3342931689 | | Yi-VL | 38 | Not recorded | 6a264c8f90966ade919bea04 | | LLaVA-v1.5 | 32.7 | Not recorded | 6a264c8f90966ade919bea04 | | OMG-LLaVA | 32.7 | Not recorded | 6a264c8f90966ade919bea04 | | LLaVA-Med | 30 | Not recorded | 6a264c8f90966ade919bea04 | | Med-Flamingo | 28.7 | Not recorded | 6a264c8f90966ade919bea04 | | MedPLIB | 28.7 | Not recorded | 6a264c8f90966ade919bea04 | | BLIP-2 | 27.3 | Not recorded | 6a264c8f90966ade919bea04 | | InstructBLIP | 25.3 | Not recorded | 6a264c8f90966ade919bea04 | | LLaVA-NeXT-7B | 24.7 | Not recorded | 692d2110d6b0dd3342931689 | | LLaVA-7B | 23.3 | Not recorded | 692d2110d6b0dd3342931689 | ### M3LLM-8B Average Score: 62.7. Result ID: 6995e51a3badfaffe6ed0710. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### Lingshu-7B Average Score: 60.7. Result ID: 6a266525fbc373592c2285b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"7B","# Data":"7.1M","Evaluation Protocol":"Partially trained on evaluation benchmarks"}. This record does not assert independent reproduction. ### InternVL3-8B Average Score: 57.3. Result ID: 6995e51a3badfaffe6ed070a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### QWen2.5-VL-7B Average Score: 54.7. Result ID: 6995e51a3badfaffe6ed0709. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. ### Lingshu-7B Average Score: 54. Result ID: 6995e51a3badfaffe6ed070d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. ### MEDSIGHT Average Score: 51.3. Result ID: 6a266525fbc373592c2285b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Unified","# Params":"8B","# Data":"72K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### HuatuoGPT-Vision-7B Average Score: 50.7. Result ID: 6995e51a3badfaffe6ed070c. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. ### MedGemma-27B Average Score: 49.3. Result ID: 6995e51a3badfaffe6ed070f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"27B"}. This record does not assert independent reproduction. ### HealthGPT-L14 Average Score: 49.2. Result ID: 6a266525fbc373592c2285b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"14B","# Data":"1.5M","Evaluation Protocol":"Partially trained on evaluation benchmarks"}. This record does not assert independent reproduction. ### HealthGPT-14B Average Score: 48. Result ID: 6995e51a3badfaffe6ed070e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"14B"}. This record does not assert independent reproduction. ### Qwen3-VL Average Score: 46.7. Result ID: 6a266525fbc373592c2285ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"8B","# Data":"-","Evaluation Protocol":"Partially trained on evaluation benchmarks"}. This record does not assert independent reproduction. ### InternVL3.5 Average Score: 46. Result ID: 6a266525fbc373592c2285af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"8B","# Data":"16.3M","Evaluation Protocol":"Partially trained on evaluation benchmarks"}. This record does not assert independent reproduction. ### InternVL2 Average Score: 43.3. Result ID: 6a266525fbc373592c2285a9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"8B","# Data":"7.3M","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### HuatuoGPT-Vision Average Score: 43.3. Result ID: 6a266525fbc373592c2285ad. [Reporting paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"7B","# Data":"647K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### HealthGPT-M3 Average Score: 43.3. Result ID: 6a266525fbc373592c2285b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"3.8B","# Data":"1.5M","Evaluation Protocol":"Partially trained on evaluation benchmarks"}. This record does not assert independent reproduction. ### Llama-3.2 Average Score: 39.3. Result ID: 6a266525fbc373592c2285aa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"11B","# Data":"-","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### LLaVA-Med-7B Average Score: 38.7. Result ID: 6995e51a3badfaffe6ed070b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. ### Yi-VL Average Score: 38. Result ID: 6a266525fbc373592c2285a8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"6B","# Data":"10K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### LLaVA-v1.5 Average Score: 32.7. Result ID: 6a266525fbc373592c2285a6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"7B","# Data":"158K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### OMG-LLaVA Average Score: 32.7. Result ID: 6a266525fbc373592c2285b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Unified","# Params":"7B","# Data":"1.2M","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### LLaVA-Med Average Score: 30. Result ID: 6a266525fbc373592c2285ac. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"7B","# Data":"60K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### Med-Flamingo Average Score: 28.7. Result ID: 6a266525fbc373592c2285ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"8.3B","# Data":"1.3M","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### MedPLIB Average Score: 28.7. Result ID: 6a266525fbc373592c2285b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Unified","# Params":"14B/7B","# Data":"500K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### BLIP-2 Average Score: 27.3. Result ID: 6a266525fbc373592c2285a5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"6.7B","# Data":"-","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### InstructBLIP Average Score: 25.3. Result ID: 6a266525fbc373592c2285a7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Comprehension only","# Params":"7B","# Data":"364K","Evaluation Protocol":"Zero-shot inference"}. This record does not assert independent reproduction. ### LLaVA-NeXT-7B Average Score: 24.7. Result ID: 6995e51a3badfaffe6ed0708. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. ### LLaVA-7B Average Score: 23.3. Result ID: 6995e51a3badfaffe6ed0707. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.