PMC-VQA
Benchmarks
TaskDatasetSOTA resultTrendResults
PMC-VQA
65.8Accuracy
103
PMC-VQA (test)
89.3Accuracy
50
PMC-VQA
63.05Accuracy
40
PMC-VQA (test)
84.6Accuracy
36
PMC-VQA
58.75Accuracy
28
PMC-VQA (test)
64.9Accuracy
27
PMC-VQA (test)
36.8Accuracy
23
PMC-VQA (test-initial)
35.4BLEU-1
19
PMC-VQA
64.35Accuracy
12
PMC-VQA OOD
54.3Accuracy
12
PMC-VQA
70.3Accuracy
12
PMC-VQA
3,818Token Cost (tokens/question)
11
PMC-VQA OOD (test)
0.058ECE
10
PMC-VQA
59.28Accuracy
8
PMC-VQA ID (test)
0.07ECE
5
PMC-VQA (300 held-out samples)
0.604Risk
5
PMC-VQA (test)
38.1Accuracy
5
PMC-VQA
0.643METEOR
4
PMC-VQA
75.2ROUGE-L F1
4
PMC-VQA
84.6COMET Score
4
PMC-VQA
0.37Sentence BLEU-4
4
PMC-VQA
89.9BERTScore
4
PMC-VQA
0.738Sentence BLEU-1
4
PMC-VQA
70.15Pass@1
4