Medical Reasoning on MedMCQA
86AccuracyReConcile
Evaluation Results
| Method | Links | |
|---|---|---|
| ReConcileBase Model=GPT-4o2025.08 | 86 | |
| MedAgentsBase Model=GPT-4o2025.08 | 85.7 | |
| TMA-AllComponBase Model=GPT-4o2025.08 | 85.4 | |
| DyLANBase Model=GPT-4o2025.08 | 84.6 | |
| MDAgentsBase Model=GPT-4o2025.08 | 80.8 | |
| GPT-4.1Access=Close-source2025.07 | 80.58 | |
| GPT-4oAccess=Close-source2025.07 | 76.17 | |
| Claude-3.5-SonnetAccess=Close-source2025.07 | 76 | |
| SMCSCategory=Ours2025.07 | 75.75 | |
| HuatuoGPT-o1-72BAccess=Open-source, Parameters=72B2025.07 | 75.25 | |
| Open-source Upper BoundCategory=Baselines2025.07 | 75.25 | |
| GPT-o3-miniAccess=Close-source2025.07 | 74.92 | |
| Symbolic-MoE*Category=Other Methods2025.07 | 74.88 | |
| Self-MoACategory=Other Methods2025.07 | 74.88 | |
| Self Consistency (Best on Validation)Category=Other Methods2025.07 | 74.83 | |
| Claude-3.7-SonnetAccess=Close-source2025.07 | 74.75 | |
| Majority VotingCategory=Other Methods2025.07 | 73.33 | |
| MoACategory=Other Methods2025.07 | 73.08 | |
| DeepSeek-R1-Distill-Llama-70BAccess=Open-source, Parameters=70B2025.07 | 72.5 | |
| Simple RouterCategory=Other Methods2025.07 | 72.5 | |
| Llama-3.3-70B-InstructAccess=Open-source, Parameters=70B2025.07 | 72.25 | |
| Qwen3-32BAccess=Open-source, Parameters=32B2025.07 | 72.17 | |
| Llama-3.3-Nemotron-Super-49B-v1Access=Open-source, Parameters=49B2025.07 | 70.92 | |
| GLM-Z1-32B-0414Access=Open-source, Parameters=32B2025.07 | 70.5 | |
| QwQ-32BAccess=Open-source, Parameters=32B2025.07 | 69.83 | |
| Qwen-2.5-72B-InstructAccess=Open-source, Parameters=72B2025.07 | 69.08 | |
| DeepSeek-R1-Distill-Qwen-32BAccess=Open-source, Parameters=32B2025.07 | 67.25 | |
| Gemma-3-27b-itAccess=Open-source, Parameters=27B2025.07 | 64.58 | |
| TMA-AllComponBase Model=MedGemma-4B2025.08 | 63.6 | |
| Qwen2.5-32b-InstructAccess=Open-source, Parameters=32B2025.07 | 62.92 | |
| Relational AggregationCategory=Ours2026.05 | 62.83 | |
| Anchor (Refined Belief)Category=Ours2026.05 | 61.12 | |
| GoAMaxCategory=Multi-Agent2026.05 | 60.04 | |
| Anchor (Initial Belief)Category=Ours2026.05 | 59.23 | |
| EXAONE-Deep-32BAccess=Open-source, Parameters=32B2025.07 | 59.17 | |
| Majority (Refined Belief)Category=Ours2026.05 | 58.87 | |
| Middle PerplexityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 58.5 | |
| OursModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 58.1 | |
| S2LModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 58 | |
| GoAMeanCategory=Multi-Agent2026.05 | 57.92 | |
| RandomModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 57.7 | |
| Embedding DiversityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 57.5 | |
| Majority (Initial Belief)Category=Ours2026.05 | 57.47 | |
| m1kModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 57.3 | |
| Qwen2.5-Coder-32B-InstructAccess=Open-source, Parameters=32B2025.07 | 57.25 | |
| LearnabilityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 57.1 | |
| TeleChat2-35B-32KAccess=Open-source, Parameters=35B2025.07 | 57.08 | |
| HIPPOTraining Paradigm=Ours2026.06 | 57 | |
| SP3FTraining Paradigm=Reward-Shaping2026.06 | 56 | |
| SCCategory=Multi-Agent2026.05 | 55.7 | |
| Self-MoACategory=Multi-Agent2026.05 | 55.56 | |
| GeneralCategory=Single-Agent2026.05 | 55.22 | |
| Qwen2.5_7BTraining Paradigm=Standard Training, Configuration=+ RL2026.06 | 55.1 | |
| RefineCategory=Multi-Agent2026.05 | 54.94 | |
| MoACategory=Multi-Agent2026.05 | 54.94 | |
| ReConcileCategory=Multi-Agent2026.05 | 54.6 | |
| Qwen2.5_7BTraining Paradigm=Standard Training2026.06 | 53.8 | |
| TMA-AllComponBase Model=Gemma-3-4B2025.08 | 53.3 | |
| DebateCategory=Multi-Agent2026.05 | 53.05 | |
| DyLANBase Model=Gemma-3-4B2025.08 | 52.4 | |
| ReConcileBase Model=Gemma-3-4B2025.08 | 52 | |
| PRM RLTraining Paradigm=Reward-Shaping2026.06 | 52 | |
| InternLM2.5-20B-ChatAccess=Open-source, Parameters=20B2025.07 | 51.92 | |
| PREF-GRPOTraining Paradigm=Reward-Shaping2026.06 | 51.8 | |
| MedAgentsBase Model=Gemma-3-4B2025.08 | 50.8 | |
| Qwen2.5_7BTraining Paradigm=Standard Training, Configuration=+ SFT2026.06 | 48.4 | |
| BiomedicalCategory=Single-Agent2026.05 | 47 | |
| CodeCategory=Single-Agent2026.05 | 45.57 | |
| MathCategory=Single-Agent2026.05 | 45.25 | |
| FinanceCategory=Single-Agent2026.05 | 42.08 | |
| LegalCategory=Single-Agent2026.05 | 41.5 | |
| MDAgentsBase Model=Gemma-3-4B2025.08 | 38.5 |