Open-ended Visual Question Answering on PMC-VQA (test-initial)
35.4BLEU-1MedVInT-TE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MedVInT-TELanguage Backbone=LLAMA-ENC [54], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 35.4 | 18.2 | |
| MedVInT-TELanguage Backbone=PubMedBERT [20], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 35.2 | 22 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC [57], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 34.8 | 18.1 | |
| MedVInT-TELanguage Backbone=PubMedBERT [20], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 33.7 | 20.4 | |
| MedVInT-TELanguage Backbone=PubMedBERT [20], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 33.7 | 20.4 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC [57], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 33 | 16.6 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA [57], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 32.7 | 20.3 | |
| MedVInT-TELanguage Backbone=PMC-LLAMA-ENC [57], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 32.6 | 16.2 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC [54], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 32.5 | 15.3 | |
| MedVInT-TELanguage Backbone=LLAMA-ENC [54], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 32.3 | 15.6 | |
| MedVInT-TDLanguage Backbone=LLAMA [54], Vision Backbone=PMC-CLIP [32], Evaluation Protocol=Trained on PMC-VQA2023.05 | 31.9 | 20 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA [57], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 31.4 | 19.5 | |
| MedVInT-TDLanguage Backbone=LLAMA [54], Vision Backbone=CLIP [47], Evaluation Protocol=Trained on PMC-VQA2023.05 | 31.3 | 19.5 | |
| MedVInT-TDLanguage Backbone=LLAMA [54], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 29.1 | 17.4 | |
| MedVInT-TDLanguage Backbone=PMC-LLAMA [57], Vision Backbone=Scratch, Evaluation Protocol=Trained on PMC-VQA2023.05 | 28.6 | 16.8 | |
| LLaVA-MedLanguage Backbone=Vicuna [14], Vision Backbone=BioMedCLIP [64], Evaluation Protocol=Zero-shot2023.05 | 28.3 | 3.7 | |
| Open-FlamingoLanguage Backbone=LLAMA [54], Vision Backbone=CLIP [47], Evaluation Protocol=Zero-shot2023.05 | 26.1 | 4.1 | |
| LLAMALanguage Backbone=LLAMA [54], Vision Backbone=None, Evaluation Protocol=Trained on PMC-VQA2023.05 | 26.1 | 14.2 | |
| BLIP-2Language Backbone=OPT-2.7B [63], Vision Backbone=CLIP [47], Evaluation Protocol=Zero-shot2023.05 | 22.5 | 5.2 |