Medical Reasoning on MedQA
92.8AccuracyReConcile
Evaluation Results
| Method | Links | |
|---|---|---|
| ReConcileBase Model=GPT-4o2025.08 | 92.8 | |
| Med-Gemini2024.04 | 91.1 | |
| TMA-AllComponBase Model=GPT-4o2025.08 | 90.7 | |
| MedAgentsBase Model=GPT-4o2025.08 | 90.3 | |
| Previous SoTA (Nori et al. 2023)2024.04 | 90.2 | |
| Best GPT-4 Method2024.04 | 90.2 | |
| DeepSeek-V42026.05 | 88.92 | |
| MDAgentsBase Model=GPT-4o2025.08 | 88.7 | |
| GPT-5.02026.05 | 87.43 | |
| DyLANBase Model=GPT-4o2025.08 | 86.8 | |
| MedGuideX-9BBackbone=Qwen3.5-9B2026.05 | 80.13 | |
| Kimi-2.62026.05 | 79.26 | |
| MedGuideX-4BBackbone=Qwen3.5-4B2026.05 | 77.06 | |
| RL with CPG-Derived Process RewardsBackbone=Qwen3.5-9B2026.05 | 75.73 | |
| Qwen3.5-9BPrompting Strategy=3-Shot In-Context Learning2026.05 | 74.94 | |
| Qwen3.5-9BPrompting Strategy=RAG with Guidelines2026.05 | 74.71 | |
| Qwen3.5-4BPrompting Strategy=RAG with Guidelines2026.05 | 73.45 | |
| Qwen3.5-9BPrompting Strategy=Zero-shot2026.05 | 73.21 | |
| Hulu-Med-7B2026.05 | 72.98 | |
| Qwen3.5-4BPrompting Strategy=Zero-shot2026.05 | 72.43 | |
| Huatuo-o1Backbone=LLaMA-3.1-8B-Instruct2026.02 | 72.4 | |
| OursModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 72.3 | |
| Qwen3.5-4BPrompting Strategy=3-Shot In-Context Learning2026.05 | 71.41 | |
| HuatuoGPT-o1-8B2026.05 | 71.09 | |
| MedReason-8B2026.05 | 70.86 | |
| m1kModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 70.4 | |
| RandomModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 69 | |
| Middle PerplexityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 68.6 | |
| S2LModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 68.5 | |
| MedGemma-1.5-4B2026.05 | 67.79 | |
| Embedding DiversityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 67.3 | |
| MedVerseBackbone=LLaMA-3.1-8B-Instruct2026.02 | 66.4 | |
| LearnabilityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 65.9 | |
| Fine-tuning with CPGBackbone=Qwen3.5-9B2026.05 | 65.04 | |
| Llama-Aloe-Beta-8B2026.05 | 64.65 | |
| Lingshu-7B2026.05 | 64.49 | |
| MedReasonBackbone=LLaMA-3.1-8B-Instruct2026.02 | 63.9 | |
| TMA-AllComponBase Model=MedGemma-4B2025.08 | 62.1 | |
| HIPPOTraining Paradigm=Ours2026.06 | 61.7 | |
| Qwen2.5_7BTraining Paradigm=Standard Training, Configuration=+ RL2026.06 | 61.2 | |
| SP3FTraining Paradigm=Reward-Shaping2026.06 | 61.1 | |
| CPGPromptBackbone=Qwen3.5-9B2026.05 | 59.15 | |
| Original (LLaMA-3.1-8B-Instruct)Backbone=LLaMA-3.1-8B-Instruct2026.02 | 58.7 | |
| MedVerseBackbone=Qwen2.5-7B-Instruct2026.02 | 58.6 | |
| PRM RLTraining Paradigm=Reward-Shaping2026.06 | 57.7 | |
| PREF-GRPOTraining Paradigm=Reward-Shaping2026.06 | 56.6 | |
| Original (Qwen2.5-7B-Instruct)Backbone=Qwen2.5-7B-Instruct2026.02 | 56.2 | |
| MedReasonBackbone=Qwen2.5-7B-Instruct2026.02 | 56.2 | |
| MediPhi-Instruct-4B2026.05 | 53.34 | |
| Clinical-R1-3B2026.05 | 52.55 | |
| Qwen2.5_7BTraining Paradigm=Standard Training, Configuration=+ SFT2026.06 | 49.5 | |
| Qwen2.5_7BTraining Paradigm=Standard Training2026.06 | 49.4 | |
| DyLANBase Model=Gemma-3-4B2025.08 | 49.2 | |
| ReConcileBase Model=Gemma-3-4B2025.08 | 48.5 | |
| Claude-Haiku-4.52026.05 | 48.08 | |
| MedAgentsBase Model=Gemma-3-4B2025.08 | 46.8 | |
| BioMistral-7B2026.05 | 45.8 | |
| TMA-AllComponBase Model=Gemma-3-4B2025.08 | 45.5 | |
| Llava-Med-v1.5-7B2026.05 | 44.78 | |
| MDAgentsBase Model=Gemma-3-4B2025.08 | 39.6 | |
| MedAlpaca-7B2026.05 | 38.81 |