Clinical Question Answering on NEJM-MedQA
86.7AccuracySAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SAGBackbone=Llama-3B each, Optimization=w/ CTDE2026.02 | 86.7 | 10.2 | |
| SAGBackbone=Qwen-4B each, Optimization=w/ GRPO2026.02 | 85.6 | 7 | |
| SAGBackbone=Qwen-4B each, Optimization=w/ CTDE2026.02 | 85.6 | 5.4 | |
| SAGBackbone=Llama-3B each, Optimization=w/ GRPO2026.02 | 84 | 6.3 | |
| Single giant LLMBackbone=Qwen-72B, Optimization=w/ DPO2026.02 | 72.5 | 15.1 | |
| SAGBackbone=Qwen-4B each, Optimization=Pre-trained2026.02 | 68.1 | 17.9 | |
| SAGBackbone=Llama-3B each, Optimization=Pre-trained2026.02 | 64.9 | 19.7 | |
| Single giant LLMBackbone=Qwen-72B, Optimization=w/ PPO2026.02 | 57.9 | 19.3 | |
| Single giant LLMBackbone=Qwen-72B, Optimization=Pre-trained2026.02 | 55.7 | 30.9 | |
| Single giant LLMBackbone=Llama-70B, Optimization=w/ PPO2026.02 | 54.6 | 18.8 | |
| Single giant LLMBackbone=Llama-70B, Optimization=w/ DPO2026.02 | 50.8 | 30.7 | |
| Me-LLaMAModel Type=Clinical specialist2026.02 | 44.2 | 26.9 | |
| Single giant LLMBackbone=Llama-70B, Optimization=Pre-trained2026.02 | 42.4 | 17.4 | |
| MeditronModel Type=Clinical specialist2026.02 | 28.5 | 28.3 |