Multiple-choice Visual Question Answering on PMC-VQA (test)
89.3AccuracyGPT-4-Oracle
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4-OracleLanguage Backbone=GPT-4, Vision Backbone=None, Protocol=Language-only2023.05 | 89.3 | |
| UMed-LVLM2025.01 | 42.6 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 40.3 | |
| MedVInT-TDLanguage Backbone=LLAMA, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 39.5 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA [57], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 39.4 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 39.2 | |
| MedVInT-TDLanguage Backbone=LLAMA, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 39.2 | |
| MedVInT2025.01 | 39.2 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 38.5 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC [57], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 38.2 | |
| MedVInT-TDLanguage Backbone=LLAMA [54], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 38.2 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 38 | |
| MedVInT-TDLanguage Backbone=LLAMA, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 37.9 | |
| MedVInT-TELanguage Backbone=PubMedBERT, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 37.6 | |
| MedVInT-TDLanguage Backbone=LLAMA [54], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 37.3 | |
| MedVInT-TELanguage Backbone=PubMedBERT [20], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 37.1 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC [57], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 37.1 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 37.1 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC [57], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 37 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC [54], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 36.9 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 36.9 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 36.9 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA [57], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 36.8 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA [57], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 36.8 | |
| MedVInT-TDLanguage Backbone=LLAMA [54], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 36.2 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 36.1 | |
| MedICap-PMCVQA-GPT-4Language Backbone=GPT-4, Vision Backbone=MedICap-PMCVQA, Protocol=Trained on PMC-VQA2023.05 | 35.9 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC [54], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 35.3 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC [54], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 35.2 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 35.2 | |
| MedVInT-TELanguage Backbone=PubMedBERT, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 34.9 | |
| LLAVA-MedLanguage Backbone=Vicuna, Vision Backbone=BioMedCLIP, Protocol=Zero-shot2023.05 | 34.8 | |
| LLAVA-Med2025.01 | 34.8 | |
| MedVInT-TELanguage Backbone=PubMedBERT [20], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 34.5 | |
| MedVInT-TELanguage Backbone=PubMedBERT [20], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 34.4 | |
| MedVInT-TELanguage Backbone=PubMedBERT, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 34.3 | |
| LLaVA-MedLanguage Backbone=Vicuna [14], Vision Backbone=BioMedCLIP [64], Evaluation Protocol=Zero-shot2023.05 | 32.6 | |
| LLAMALanguage Backbone=LLAMA, Vision Backbone=None, Protocol=Language-only2023.05 | 30.8 | |
| LLAMALanguage Backbone=LLAMA [54], Vision Backbone=None, Evaluation Protocol=Trained on PMC-VQA2023.05 | 30.6 | |
| MedICap-GPT-4Language Backbone=GPT-4, Vision Backbone=MedICap, Protocol=Zero-shot2023.05 | 27.2 | |
| Open-FlamingoLanguage Backbone=LLAMA, Vision Backbone=CLIP, Protocol=Zero-shot2023.05 | 26.4 | |
| Open-Flamingo2025.01 | 26.4 | |
| GPT-4Language Backbone=GPT-4, Vision Backbone=None, Protocol=Language-only2023.05 | 25.7 | |
| Open-FlamingoLanguage Backbone=LLAMA [54], Vision Backbone=CLIP [47], Evaluation Protocol=Zero-shot2023.05 | 25 | |
| PMC-CLIPLanguage Backbone=PMC-CLIP, Vision Backbone=PMC-CLIP, Protocol=Zero-shot2023.05 | 24.7 | |
| PMC-CLIP2025.01 | 24.7 | |
| BLIP-2Language Backbone=OPT-2.7B [63], Vision Backbone=CLIP [47], Evaluation Protocol=Zero-shot2023.05 | 24.6 | |
| BLIP-2Language Backbone=OPT-2.7B, Vision Backbone=CLIP, Protocol=Zero-shot2023.05 | 24.3 | |
| BLIP-22025.01 | 24.3 | |
| PMC-CLIPLanguage Backbone=PMC-CLIP [32], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Zero-shot2023.05 | 24 |