Clinical Question Answering on MedQA
94.2AccuracyDeepSeek-R1
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeek-R1Backbone=DeepSeek-R12026.07 | 94.2 | — | |
| Claude Sonnet 4.5Backbone=Claude Sonnet 4.52026.07 | 93 | — | |
| SAGBackbone=Qwen-4B each, Optimization=w/ GRPO2026.02 | 91.4 | 7 | |
| SAGBackbone=Qwen-4B each, Optimization=w/ CTDE2026.02 | 91 | 5.4 | |
| SAGBackbone=Llama-3B each, Optimization=w/ GRPO2026.02 | 90.3 | 6.3 | |
| SAGBackbone=Llama-3B each, Optimization=w/ CTDE2026.02 | 89.6 | 10.2 | |
| SAGBackbone=Qwen-4B each, Optimization=Pre-trained2026.02 | 86 | 17.9 | |
| SAGBackbone=Llama-3B each, Optimization=Pre-trained2026.02 | 84.6 | 19.7 | |
| MedGemma 27BBackbone=MedGemma 27B2026.07 | 83 | — | |
| Single giant LLMBackbone=Qwen-72B, Optimization=w/ DPO2026.02 | 82.4 | 15.1 | |
| Single giant LLMBackbone=Llama-70B, Optimization=w/ DPO2026.02 | 81.5 | 30.7 | |
| Qwen3 8BBackbone=Qwen3 8B2026.07 | 79.4 | — | |
| Single giant LLMBackbone=Qwen-72B, Optimization=w/ PPO2026.02 | 77.2 | 19.3 | |
| FaithMed (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=FaithMed, Training Recipe=no-SFT recipe2026.07 | 74.8 | — | |
| Single giant LLMBackbone=Llama-70B, Optimization=w/ PPO2026.02 | 73.4 | 18.8 | |
| Single giant LLMBackbone=Qwen-72B, Optimization=Pre-trained2026.02 | 72 | 30.9 | |
| Agentic Search + GRPO (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=Agentic Search + GRPO, Training Recipe=GRPO (outcome rewards only)2026.07 | 71.3 | — | |
| Agentic Search (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=Agentic Search2026.07 | 70.6 | — | |
| MedRAG (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=MedRAG2026.07 | 70.2 | — | |
| Base (Qwen3 4B)Backbone=Qwen3 4B, Method Variant=Base2026.07 | 69.3 | — | |
| Gemma 3 12BBackbone=Gemma 3 12B2026.07 | 69 | — | |
| Llama 3.1 8B InstructBackbone=Llama 3.1 8B, Method Variant=Instruct2026.07 | 61.8 | — | |
| FaithMed (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=FaithMed, Training Recipe=no-SFT recipe2026.07 | 61.4 | — | |
| Single giant LLMBackbone=Llama-70B, Optimization=Pre-trained2026.02 | 59.8 | 17.4 | |
| Me-LLaMAModel Type=Clinical specialist2026.02 | 58 | 26.9 | |
| MedRAG (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=MedRAG2026.07 | 51.4 | — | |
| Agentic Search + GRPO (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=Agentic Search + GRPO, Training Recipe=GRPO (outcome rewards only)2026.07 | 50.9 | — | |
| Agentic Search (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=Agentic Search2026.07 | 49.2 | — | |
| Base (Qwen3 1.7B)Backbone=Qwen3 1.7B, Method Variant=Base2026.07 | 48.9 | — | |
| MeditronModel Type=Clinical specialist2026.02 | 34.9 | 28.3 |