Question Answering on MedMCQA (Accuracy)
80AccuracyClaude Sonnet 4.5
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude Sonnet 4.5Backbone=Claude Sonnet 4.52026.07 | 80 | |
| MedGemma 27BBackbone=MedGemma 27B2026.07 | 73 | |
| DeepSeek-R1Backbone=DeepSeek-R12026.07 | 72.8 | |
| Qwen3 8BBackbone=Qwen3 8B2026.07 | 66.3 | |
| MA-PoP2026.05 | 64.67 | |
| TARSEBackbone=14B2026.03 | 64.3 | |
| FaithMed (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=FaithMed2026.07 | 64.1 | |
| ISP2026.05 | 63.67 | |
| Agentic Search (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=Agentic Search2026.07 | 63.4 | |
| LLP2026.05 | 63.21 | |
| Agentic Search + GRPO (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=Agentic Search + GRPO2026.07 | 63.2 | |
| Free MADT=52026.05 | 62.67 | |
| TARSEBackbone=7B2026.03 | 62.1 | |
| Decentr. MADT=22026.05 | 62 | |
| Sparse MADT=22026.05 | 62 | |
| Free MADT=32026.05 | 62 | |
| AttributionIteration=1, Selection Ratio=0.22026.04 | 61.92 | |
| EVOSELECTIteration=2, Selection Ratio=0.52026.04 | 61.87 | |
| RandomIteration=1, Selection Ratio=0.22026.04 | 61.8 | |
| Attr-DivIteration=1, Selection Ratio=0.52026.04 | 61.8 | |
| EVOSELECTIteration=1, Selection Ratio=0.52026.04 | 61.73 | |
| Sparse MADT=52026.05 | 61.67 | |
| Majority Voting2026.05 | 61.67 | |
| Self-Consistency2026.05 | 61.67 | |
| Attr-DivIteration=1, Selection Ratio=0.22026.04 | 61.58 | |
| AllIteration=22026.04 | 61.58 | |
| Attr-DivIteration=2, Selection Ratio=0.52026.04 | 61.58 | |
| EVOSELECTIteration=2, Selection Ratio=0.22026.04 | 61.58 | |
| DiversityIteration=1, Selection Ratio=0.52026.04 | 61.56 | |
| AttributionIteration=2, Selection Ratio=0.52026.04 | 61.53 | |
| AttributionIteration=2, Selection Ratio=0.22026.04 | 61.46 | |
| AttributionIteration=1, Selection Ratio=0.52026.04 | 61.44 | |
| DiversityIteration=2, Selection Ratio=0.52026.04 | 61.44 | |
| AllIteration=12026.04 | 61.39 | |
| RandomIteration=1, Selection Ratio=0.52026.04 | 61.39 | |
| TSDSIteration=1, Selection Ratio=0.52026.04 | 61.39 | |
| Free MADT=22026.05 | 61.33 | |
| DiversityIteration=1, Selection Ratio=0.22026.04 | 61.32 | |
| RandomIteration=2, Selection Ratio=0.22026.04 | 61.32 | |
| RandomIteration=2, Selection Ratio=0.52026.04 | 61.32 | |
| DiversityIteration=2, Selection Ratio=0.22026.04 | 61.22 | |
| Attr-DivIteration=2, Selection Ratio=0.22026.04 | 61.22 | |
| EVOSELECTIteration=1, Selection Ratio=0.22026.04 | 61.2 | |
| TSDSIteration=2, Selection Ratio=0.52026.04 | 61.13 | |
| TSDSIteration=1, Selection Ratio=0.22026.04 | 61.01 | |
| Base2026.04 | 60.75 | |
| Decentr. MADT=32026.05 | 60.67 | |
| TSDSIteration=2, Selection Ratio=0.22026.04 | 60.34 | |
| Sparse MADT=32026.05 | 60.33 | |
| GoA_MaxAgent Configuration=Multi-Agent, Number of Agents=3, Zero-shot CoT=true2026.04 | 60.04 | |
| Decentr. MADT=52026.05 | 60 | |
| MedRAG (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=MedRAG2026.07 | 59.8 | |
| Base (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=Base2026.07 | 59.7 | |
| Centr. MADT=52026.05 | 59.67 | |
| Centr. MADT=22026.05 | 59 | |
| Gemma 3 12BBackbone=Gemma 3 12B2026.07 | 59 | |
| Centr. MADT=32026.05 | 58.33 | |
| GoA_MeanAgent Configuration=Multi-Agent, Number of Agents=3, Zero-shot CoT=true2026.04 | 57.92 | |
| Falcon-7B2026.05 | 57.67 | |
| i-MedRAG-Qwen2.5Backbone=7B2026.03 | 57.6 | |
| Llama 3.1 8B InstructBackbone=Llama 3.1 8B2026.07 | 57.3 | |
| MedRAG-Qwen2.5Backbone=7B2026.03 | 56.9 | |
| FaithMed (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=FaithMed2026.07 | 56.9 | |
| rStar-Qwen2.5Backbone=14B2026.03 | 56.7 | |
| rStar-Qwen2.5Backbone=7B2026.03 | 55.9 | |
| Qwen2.5-InstructBackbone=14B2026.03 | 55.7 | |
| SCAgent Configuration=Multi-Agent, Number of Agents=6, Zero-shot CoT=true2026.04 | 55.7 | |
| Gemma-9B2026.05 | 55.67 | |
| Self-MoAAgent Configuration=Multi-Agent, Number of Agents=6, Zero-shot CoT=true2026.04 | 55.56 | |
| Qwen2.5-7B-InstructAgent Configuration=Single-Agent, Number of Agents=1, Zero-shot CoT=true, Agent Role/Strategy=General2026.04 | 55.22 | |
| RefineAgent Configuration=Multi-Agent, Number of Agents=6, Zero-shot CoT=true2026.04 | 54.94 | |
| MoAAgent Configuration=Multi-Agent, Number of Agents=6, Zero-shot CoT=true2026.04 | 54.94 | |
| Qwen-7B2026.05 | 54.67 | |
| ReConcileAgent Configuration=Multi-Agent, Number of Agents=6, Zero-shot CoT=true2026.04 | 54.6 | |
| Qwen2.5-InstructBackbone=7B2026.03 | 54.1 | |
| DebateAgent Configuration=Multi-Agent, Number of Agents=6, Zero-shot CoT=true2026.04 | 53.05 | |
| MedRAG (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=MedRAG2026.07 | 51.3 | |
| Agentic Search + GRPO (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=Agentic Search + GRPO2026.07 | 50.1 | |
| Llama-8B2026.05 | 50 | |
| TSDSIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 49.39 | |
| EVOSELECTIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 49.27 | |
| EVOSELECTIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 49.15 | |
| TSDSIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.98 | |
| EVOSELECTIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.94 | |
| RandomIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.91 | |
| AllIteration=1, Selection Ratio=All, Backbone=3b base model2026.04 | 48.67 | |
| EVOSELECTIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.63 | |
| Attr-DivIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.55 | |
| DiversityIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.48 | |
| RandomIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.43 | |
| Attr-DivIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.29 | |
| RandomIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.24 | |
| Attr-DivIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.2 | |
| AttributionIteration=1, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.17 | |
| DiversityIteration=2, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.17 | |
| AttributionIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.1 | |
| AttributionIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.08 | |
| TSDSIteration=1, Selection Ratio=0.2, Backbone=3b base model2026.04 | 48.08 | |
| TSDSIteration=2, Selection Ratio=0.5, Backbone=3b base model2026.04 | 48.03 | |
| Base (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=Base2026.07 | 48 |