MedQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MedQA-USMLE
73.61Accuracy
5
MedQA
68.1Accuracy (IT)
5
MedQA (test)
97.5QA Accuracy
5
MedQA in-domain
99.2Accuracy
5
MedQA (val)
81Accuracy
4
MedQA Biomedical
54.1Accuracy
4
MedQA OOD (out-of-distribution)
57.5MCQ Accuracy
4
MedQA (test)
31.26Accuracy
4
MedQA
71EM
4
MedQA
74.11Accuracy
4
MedQA-USMLE
79.8Accuracy (Letter Match)
4
MedQA
92.8Accuracy
4
MedQA
4.3nAUPC
4
MedQA
100Success Rate (SR)
4
MedQA
49.2Adaptivity
4
MedQA USMLE
97.88Pass@1
4
MedQA MCMLE
96.21Pass@1
4
MedQA US
85.9Accuracy
4
MedQA Mainland China
86Accuracy
4
MedQA Taiwan
91Accuracy
4
MedQA first 100 questions (test)
99Accuracy (Web Search On)
3
MedQA
0Risk
3
MedQA 80/20 split (seed 42) (val)
6.9Risk
3
MedQA USMLE
9.4Exclusive Catch Rate (@10%)
3
MedQA diabetes
88Accuracy
3