Open-ended Visual Question Answering on PMC-VQA (test)
36.8AccuracyMedVInT-TE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MedVInT-TELanguage Backbone=LLAMA-ENC, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 36.8 | 18.4 | |
| MedVInT-TELanguage Backbone=PubMedBERT, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 36.4 | 23.2 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 35.3 | 18.6 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 35 | 17 | |
| MedVInT-TELanguage Backbone=PubMedBERT, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 34.4 | 20.8 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 34.4 | 16.5 | |
| MedVInT-TELanguage Backbone=PubMedBERT, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 34.2 | 20.9 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 33.6 | 21.5 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 33.4 | 15.1 | |
| MedVInT-TDLanguage Backbone=LLAMA, Vision Backbone=PMC-CLIP, Protocol=Trained on PMC-VQA2023.05 | 33.4 | 21.3 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 32.6 | 20.4 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 32.5 | 15.9 | |
| MedVInT-TDLanguage Backbone=LLAMA, Vision Backbone=CLIP, Protocol=Trained on PMC-VQA2023.05 | 32.2 | 20 | |
| MedVInT-TDLanguage Backbone=LLAMA, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 30.2 | 18 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA, Vision Backbone=Scratch, Protocol=Trained on PMC-VQA2023.05 | 29.8 | 17.4 | |
| LLAVA-MedLanguage Backbone=Vicuna, Vision Backbone=BioMedCLIP, Protocol=Zero-shot2023.05 | 29.4 | 3.9 | |
| LLAMALanguage Backbone=LLAMA, Vision Backbone=None, Protocol=Language-only2023.05 | 27.2 | 14.6 | |
| Open-FlamingoLanguage Backbone=LLAMA, Vision Backbone=CLIP, Protocol=Zero-shot2023.05 | 26.5 | 4.1 | |
| MedICap-PMCVQA-GPT-4Language Backbone=GPT-4, Vision Backbone=MedICap-PMCVQA, Protocol=Trained on PMC-VQA2023.05 | 22.4 | 3.8 | |
| GPT-4-OracleLanguage Backbone=GPT-4, Vision Backbone=None, Protocol=Language-only2023.05 | 22 | 18.8 | |
| BLIP-2Language Backbone=OPT-2.7B, Vision Backbone=CLIP, Protocol=Zero-shot2023.05 | 21.8 | 7.6 | |
| GPT-4Language Backbone=GPT-4, Vision Backbone=None, Protocol=Language-only2023.05 | 21.1 | 3 | |
| MedICap-GPT-4Language Backbone=GPT-4, Vision Backbone=MedICap, Protocol=Zero-shot2023.05 | 20.9 | 4.2 |