MedMCQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MedMCQA In-Domain (test)
90Accuracy
10
MedMCQA 1,000-example evaluation slice
39.8Accuracy
9
MedMCQA 100 sampled
83Accuracy
9
MedMCQA
6.43FDR (%)
9
MedMCQA translated (test)
43.2Accuracy (ZH)
9
MedMCQA
50.2Agreement (%)
8
MedMCQA (test)
75.3SM Score
8
MedMCQA OOD (out-of-distribution)
66.2Accuracy
7
MedMCQA Far-Domain
84.4AUROC
7
MedMCQA
1.13KL Divergence
7
MedMCQA (val)
90Accuracy
7
MedMCQA
86.1Accuracy
6
MedMCQA dental
36.87Accuracy
6
MedMCQA
91.2Forgetting Rate
6
MedMCQA (test)
0.52MIA Score
6
MedMCQA subset
0.645AUROC
6
MedMCQA (test)
71.7AUROC
6
MedMCQA (test)
73.7Accuracy
6
MedMCQA (test)
92.5QA Accuracy
5
MedMCQA Pharmacology (val)
56.4Accuracy
4
MedMCQA (test)
41.67Accuracy
4
MedMCQA diabetes
76.5Accuracy
3
MedMCQA (test)
28.1WER
3
MedMCQA
100Accuracy under Attack
2
MedMCQA 200 samples
0.36Log-likelihood Score
2