MedMCQA
Benchmarks
TaskDatasetSOTA resultTrendResults
MedMCQA
90.4Accuracy
591
MedMCQA (test)
84.13Accuracy
134
MedMCQA
80Accuracy
125
MedMCQA
58.2Accuracy (ACC)
81
MedMCQA
86Accuracy
72
MedMCQA
10.82BLEU Score
54
MedMCQA
75.95AUC
51
MedMCQA (test)
0.163Test Error Rate
48
MedMCQA
75.57AUC
42
MedMCQA
88.9Accuracy
42
MedMCQA (fresh candidate pool)
57.4Accuracy
34
MedMCQA
63.31Accuracy
32
MedMCQA
53.6Pass@1 Accuracy
28
MedMCQA edit
51Efficacy
18
MedMCQA i.d.
82.66Accuracy
15
MedMCQA
72.33R@1
15
MedMCQA QF=1000
90Forget Accuracy
14
MEDMCQA (val)
96.3Top-1 Acc
14
MedMCQA
96.3Top-1 Acc
14
MedMCQA
86.1Accuracy
14
MedMCQA
4.3Tau Correlation
13
MedMCQA
3.28Perplexity (PPL)
12
MedMCQA
1,047Token Cost (tokens/question)
11
MedMCQA (dev)
0.791Accuracy
11
MedMCQA (val)
58.7Accuracy
10