Medical Question Answering on MedQA USMLE (Accuracy)
90.4AccuracyGemini-3-flash
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-flash2025.10 | 90.4 | |
| CoT-SCModel=LLaMA3-70b, Quantization=int42025.04 | 80 | |
| QM-ToTModel=LLaMA3-70b, Quantization=int42025.04 | 79.7 | |
| CoT-AVGModel=LLaMA3-70b, Quantization=int42025.04 | 78.65 | |
| ToTModel=LLaMA3-70b, Quantization=int42025.04 | 76.72 | |
| QM-ToTModel=Qwen2.5-72b, Quantization=int42025.04 | 74.25 | |
| ToTModel=Qwen2.5-72b, Quantization=int42025.04 | 73 | |
| CoT-AVGModel=Qwen2.5-72b, Quantization=int42025.04 | 72.37 | |
| AM-localNumber of Adapters=30, External Tools=false2025.10 | 70.6 | |
| CoT-SCModel=Qwen2.5-72b, Quantization=int42025.04 | 70.25 | |
| AM-fullNumber of Adapters=30, External Tools=true2025.10 | 70.2 | |
| Qwen3.5-9BToken Budget=generous2025.10 | 69.6 | |
| QM-ToTModel=LLaMA3.1-8b, Quantization=int42025.04 | 69.49 | |
| ToTModel=LLaMA3.1-8b, Quantization=int42025.04 | 60.77 | |
| CoT-SCModel=LLaMA3.1-8b, Quantization=int42025.04 | 59.19 | |
| CoT-AVGModel=LLaMA3.1-8b, Quantization=int42025.04 | 58.77 | |
| QM-ToTModel=LLaMA2-70b, Quantization=int42025.04 | 50.25 | |
| ToTModel=LLaMA2-70b, Quantization=int42025.04 | 35.25 | |
| CoT-AVGModel=LLaMA2-70b, Quantization=int42025.04 | 34 | |
| Qwen3.5-9BToken Budget=strict2025.10 | 28.4 | |
| CoT-SCModel=LLaMA2-70b, Quantization=int42025.04 | 27.75 |