Multiple-choice Question Answering on MedMCQA
88.9AccuracyGPT-5
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-5Family=Frontier API2026.05 | 88.9 | |
| GPT-4.1Family=Frontier API2026.05 | 83.9 | |
| GPT-4.1-miniFamily=Frontier API2026.05 | 78.4 | |
| S-DPOModel=Llama32026.05 | 72.19 | |
| MASS-DPOModel=Llama32026.05 | 71.29 | |
| DPO-kModel=Llama32026.05 | 71.04 | |
| CLR-voyance-8BFamily=8B family, Merge Strategy=DELLA-Linear2026.05 | 67 | |
| Qwen3-8B baseFamily=8B family2026.05 | 66 | |
| HuatuoGPT-o1Family=Medical / open-source2026.05 | 64.5 | |
| MG-BFamily=4B family, Merge Strategy=Breadcrumbs2026.05 | 61 | |
| CLR-voyance-4BFamily=4B family, Merge Strategy=DELLA-Linear2026.05 | 58.5 | |
| MG-Δ+AFamily=4B family, Merge Strategy=DELLA-Linear+AIM2026.05 | 58.5 | |
| MG-B+AFamily=4B family, Merge Strategy=Breadcrumbs+AIM2026.05 | 58.5 | |
| MASS-DPOModel=Qwen32026.05 | 56.66 | |
| DPO-kModel=Qwen32026.05 | 55.56 | |
| MG-4B baseFamily=4B family2026.05 | 54.5 | |
| S-DPOModel=Qwen32026.05 | 52.56 | |
| DPOModel=Llama32026.05 | 52.25 | |
| S-DPOModel=SmolLM32026.05 | 44.99 | |
| Gemma-3-4BFamily=Medical / open-source2026.05 | 44.5 | |
| MASS-DPOModel=SmolLM32026.05 | 44.19 | |
| DPO-kModel=SmolLM32026.05 | 44.09 | |
| DPOModel=Qwen32026.05 | 43.49 | |
| Nemotron-CC_ASI+Pre-training Curation Strategy=AI Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 40.97 | |
| DS-R1-0528Family=Medical / open-source2026.05 | 36 | |
| DS-R1-DistillFamily=Medical / open-source2026.05 | 33.5 | |
| DPOModel=SmolLM32026.05 | 33.27 | |
| MedGemma-27BFamily=Medical / open-source2026.05 | 33 | |
| Nemotron-CC_ASIPre-training Curation Strategy=AI Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 30.28 | |
| DMPOModel=Qwen32026.05 | 28.91 | |
| Nemotron-CCPre-training Curation Strategy=AI Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 28.86 | |
| DCLMPre-training Curation Strategy=Human Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 28.15 | |
| Fineweb-EduPre-training Curation Strategy=Human Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 25.8 | |
| DMPOModel=Llama32026.05 | 25.7 | |
| DMPOModel=SmolLM32026.05 | 25.5 | |
| Ultra-FinewebPre-training Curation Strategy=Human Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 24.92 | |
| GPT-4o2026.02 | 0.5785 | |
| Mistral Large2026.02 | 0.4344 | |
| Llama3-70B-InstructSize=70B, Type=Instruct2026.02 | 0.4167 | |
| Claude 3 Haiku2026.02 | 0.4057 | |
| Qwen1.5 (1.8B)Size=1.8B2026.02 | 0.047 | |
| Gemma (2B)Size=2B2026.02 | 0.0457 |