Question Answering on MedQA-USMLE (test)
94.34AccuracyKimi-K2-Thinking-1TB
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Kimi-K2-Thinking-1TBModel Group=Awesome General Models2026.01 | 94.34 | — | |
| Deepseek-v3.2-685BModel Group=Awesome General Models2026.01 | 93.72 | — | |
| Gemini2.5-ProModel Group=Awesome General Models2026.01 | 92.22 | — | |
| DEEPMED-14B-RLModel Group=DeepResearch Models, Training Stage=RL2026.01 | 88.22 | — | |
| Qwen3-30BA3B-ThinkingModel Group=Awesome General Models2026.01 | 88.14 | — | |
| Tongyi-DeepResearch-30BA3BModel Group=DeepResearch Models2026.01 | 87.9 | — | |
| AlphaMed-70BModel Group=Medical Reasoning Models2026.01 | 87.52 | — | |
| DEEPMED-14B-SFTModel Group=DeepResearch Models, Training Stage=SFT2026.01 | 87.27 | — | |
| Human (expert)Setting=Manual2023.08 | 87 | — | |
| Human (expert score)2022.07 | 87 | — | |
| Med-PaLM 2Prompting strategy=best2023.05 | 86.5 | — | |
| MedPalm v2Date=20232022.07 | 86.5 | — | |
| GPT-4-baseFew-shot evaluation protocol=5-shot2023.05 | 86.1 | — | |
| GPT-4Date=20232022.07 | 86.1 | — | |
| QuarkMed-32BModel Group=Medical Reasoning Models2026.01 | 86.02 | — | |
| Med-PaLM 2Prompting strategy=Ensemble Refinement (ER)2023.05 | 85.4 | — | |
| BaiChuan-M2-32BModel Group=Medical Reasoning Models2026.01 | 84.68 | — | |
| M1-1K-32BModel Group=Medical Reasoning Models2026.01 | 83.5 | — | |
| HuatuoGPT-o1-70BModel Group=Medical Reasoning Models2026.01 | 83.27 | — | |
| Qwen3-14BModel Group=Awesome General Models2026.01 | 82.17 | — | |
| GPT-4Few-shot evaluation protocol=5-shot2023.05 | 81.4 | — | |
| SFTBackbone=Qwen32026.01 | 80.8 | — | |
| GRPO2026.02 | 78.14 | 2,732 | |
| 5-shotBackbone=Qwen32026.01 | 77.17 | — | |
| ESTARoptimization=RL2026.02 | 77.13 | 388 | |
| ESTAR-FTmode=fine-tuned2026.02 | 77.1 | 645 | |
| FlashThink2026.02 | 76.83 | 1,693 | |
| ESTAR-LITEearly_stopping=classifier-based2026.02 | 76.83 | 549 | |
| kNN-MoEBackbone=Qwen32026.01 | 76.7 | — | |
| O1-Pruner2026.02 | 76.6 | 1,472 | |
| Length-Penaltyexplicit_length_penalty=true2026.02 | 76.6 | 1,325 | |
| AdaptThink2026.02 | 76.4 | 987 | |
| SFT (Router Only)Backbone=Qwen32026.01 | 76.05 | — | |
| Zero-shotBackbone=Qwen32026.01 | 75.3 | — | |
| M1-1K-7BModel Group=Medical Reasoning Models2026.01 | 71.56 | — | |
| MedResaon-8BModel Group=Medical Reasoning Models2026.01 | 68.4 | — | |
| kNN-MoEBackbone=GPT-OSS2026.01 | 68.31 | — | |
| Flan-PaLMnumber of parameters=540 B2022.12 | 67.6 | — | |
| Flan-PaLMPrompting strategy=best2023.05 | 67.6 | — | |
| Zero-shotBackbone=GPT-OSS2026.01 | 67.29 | — | |
| No-Thinkingreasoning=disabled2026.02 | 66.2 | 315 | |
| SFTBackbone=GPT-OSS2026.01 | 65.61 | — | |
| SFT (Router Only)Backbone=GPT-OSS2026.01 | 65.05 | — | |
| Llama-2 5-shot CoTDate=2023, Model Parameters=70B, Few-shot shots=5, Chain-of-Thought (CoT)=true, Ensemble samples (k)=502022.07 | 62.5 | — | |
| Codex 5-shot CoTDate=2022, Few-shot shots=5, Chain-of-Thought (CoT)=true, Ensemble samples (k)=1002022.07 | 60.2 | — | |
| 5-shotBackbone=GPT-OSS2026.01 | 60.02 | — | |
| Human (passing score)2022.07 | 60 | — | |
| ChatGPTSetting=Zero-shot2023.08 | 57 | — | |
| BioMedGPT-10BSetting=Fine-tuning2023.08 | 50.4 | — | |
| PubMedGPTnumber of parameters=2.7 B2022.12 | 50.3 | — | |
| PubMedGPTDate=2022, Evaluation Protocol=Finetuned SOTA2022.07 | 50.3 | — | |
| Human (pass)Setting=Manual2023.08 | 50 | — | |
| DRAGONBackbone=BioLinkBERT-Large2022.10 | 47.5 | — | |
| DRAGONnumber of parameters=360 M2022.12 | 47.5 | — | |
| InstructGPTSetting=Zero-shot2023.08 | 46 | — | |
| KALE2026.01 | 45.89 | — | |
| Llama2-ChatSetting=Fine-tuning2023.08 | 45.3 | — | |
| BioLinkBERT + GreaseLMBackbone=BioLinkBERT-Large, KG Module=GreaseLM2022.10 | 45.1 | — | |
| BioLinkBERTnumber of parameters=340 M2022.12 | 45.1 | — | |
| BioLinkBERT + QAGNNBackbone=BioLinkBERT-Large, KG Module=QAGNN2022.10 | 45 | — | |
| PMC-LlamaSetting=Fine-tuning2023.08 | 44.7 | — | |
| BioLinkBERTSize=large, Number of parameters=340M2022.03 | 44.6 | — | |
| BioLinkBERTBackbone=BioLinkBERT-Large2022.10 | 44.6 | — | |
| LlamaSetting=Fine-tuning2023.08 | 44.6 | — | |
| Galacticanumber of parameters=120 B2022.12 | 44.4 | — | |
| KG-SFT2026.01 | 41.71 | — | |
| AugGPT2026.01 | 40.29 | — | |
| BioLinkBERTSize=base, Number of parameters=110M2022.03 | 40 | — | |
| GPT3Mix2026.01 | 39.35 | — | |
| GreaseLM2022.03 | 38.5 | — | |
| PubmedBERTSize=base2022.03 | 38.1 | — | |
| PubmedBERT2022.10 | 38.1 | — | |
| PubMedBERTnumber of parameters=100 M2022.12 | 38.1 | — | |
| QAGNN2022.03 | 38 | — | |
| QA-GNNbackbone=SapBERT2021.04 | 38 | — | |
| SFTBackbone=OLMoE2026.01 | 37.84 | — | |
| COT2026.01 | 37.65 | — | |
| SapBERT2021.04 | 37.2 | — | |
| BioBERTSize=large2022.03 | 36.7 | — | |
| BioBERT2022.10 | 36.7 | — | |
| BioBERT-large2021.04 | 36.7 | — | |
| KAPING2026.01 | 36.39 | — | |
| kNN-MoEBackbone=OLMoE2026.01 | 35.04 | — | |
| RoBERTa-large2021.04 | 35 | — | |
| Llama2-ChatSetting=Zero-shot2023.08 | 34.4 | — | |
| BERT-base2021.04 | 34.3 | — | |
| TOG2026.01 | 34.27 | — | |
| BioBERT-base2021.04 | 34.1 | — | |
| SFT2026.01 | 33.62 | — | |
| GPT-Neonumber of parameters=2.7 B2022.12 | 33.3 | — | |
| KGR2026.01 | 33.15 | — | |
| Zero-shotBackbone=OLMoE2026.01 | 32.81 | — | |
| 5-shotBackbone=OLMoE2026.01 | 31.31 | — | |
| SFT (Router Only)Backbone=OLMoE2026.01 | 31.22 | — | |
| Vanilla2026.01 | 28.2 | — | |
| Llama2Setting=Zero-shot2023.08 | 27.2 | — | |
| LlamaSetting=Zero-shot2023.08 | 27.1 | — | |
| FoSS2026.02 | 25.27 | — | |
| CoG2026.02 | 25.06 | — | |
| Transformer (w/o FT)fine-tuned=false2026.02 | 24.89 | — |