Medical Question Answering on NEJM-MedQA
0.22Base DeviationSAG (w/ CTDE)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SAG (w/ CTDE)Backbone=Qwen2026.02 | 0.22 | 0.47 | 0.39 | 0.55 | |
| SAG (w/ CTDE)Backbone=Llama, Training Strategy=CTDE2026.02 | 0.32 | 0.46 | 0.34 | 0.51 | |
| SAG (w/ GRPO)Backbone=Qwen2026.02 | 0.74 | 1.01 | 0.95 | 1.26 | |
| SAG (w/ GRPO)Backbone=Llama, Training Strategy=GRPO2026.02 | 0.82 | 1.13 | 0.94 | 1.27 | |
| Single, giant LLM w/ PPOBackbone=Llama, Training Strategy=PPO2026.02 | 1.23 | 4.54 | 1.38 | 1.82 | |
| SAG (Pre-Trained)Backbone=Qwen2026.02 | 1.47 | 1.89 | 1.61 | 2.07 | |
| SAG (Pre-Trained)Backbone=Llama, Configuration=Pre-Trained2026.02 | 1.52 | 1.94 | 1.73 | 2.12 | |
| Single, giant LLMBackbone=Qwen2026.02 | 2.05 | 2.3 | 1.25 | 1.85 | |
| Single, giant LLM w/ DPOBackbone=Qwen2026.02 | 2.42 | 2.75 | 1.65 | 2.25 | |
| Single, giant LLM w/ PPOBackbone=Qwen2026.02 | 2.88 | 3.05 | 1.95 | 2.65 | |
| Me-LLaMABackbone=Llama2026.02 | 3.96 | 4.82 | 4.41 | 5.24 | |
| Single, giant LLMBackbone=Llama2026.02 | 4.18 | 5.14 | 4.82 | 5.47 | |
| Single, giant LLM w/ DPOBackbone=Llama, Training Strategy=DPO2026.02 | 5.02 | 6.31 | 5.94 | 6.83 |