Medical Knowledge Question Answering on Medical Domain (MedQA, MMLU, MedMCQA) (test)
92.5MedQA ScoreGPT-4.1 (25-04-14)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4.1 (25-04-14)Params=N/A, Label=false, Knowledge=false2026.01 | 92.5 | 97.3 | 81.3 | 90.4 | |
| DeepSeek-R1Params=671B, Label=false, Knowledge=false2026.01 | 91 | 99 | 78.7 | 89.6 | |
| Gemini-2.5-Pro (25-06-05)Params=N/A, Label=false, Knowledge=false2026.01 | 89.7 | 98.1 | 82.1 | 90 | |
| Med-PaLM2Params=340B, Label=true, Knowledge=false2026.01 | 85.4 | 92 | 72.3 | 83.2 | |
| Llama3.1-InstructParams=70B, Label=false, Knowledge=false2026.01 | 82.3 | 92.4 | 72.2 | 82.3 | |
| Qwen2.5-InstructParams=72B, Label=false, Knowledge=false2026.01 | 79.6 | 94.1 | 68.6 | 80.8 | |
| Qwen2.5-InstructParams=32B, Label=false, Knowledge=false2026.01 | 73.1 | 93.2 | 63.1 | 76.5 | |
| DGRC (Ours)Params=7B, Label=false, Knowledge=false2026.01 | 72.8 | 90.9 | 67.5 | 77.1 | |
| RLAIF (GRPO)Params=7B, Label=false, Knowledge=false2026.01 | 71.4 | 89.1 | 68.1 | 76.2 | |
| Baseline-w/-labelParams=7B, Label=true, Knowledge=false2026.01 | 70.1 | 90.1 | 64.5 | 74.9 | |
| Baseline-w/o-labelParams=7B, Label=false, Knowledge=false2026.01 | 69.4 | 87.2 | 64.3 | 73.7 | |
| Llama3.1-InstructParams=8B, Label=false, Knowledge=false2026.01 | 65.8 | 80.5 | 56.5 | 67.6 | |
| MegaScienceParams=7B, Label=false, Knowledge=true2026.01 | 61 | 87.1 | 57.4 | 68.5 | |
| DGRC (Ours)Params=3B, Label=false, Knowledge=false2026.01 | 60.6 | 81 | 59.7 | 67.1 | |
| RLAIF (GRPO)Params=3B, Label=false, Knowledge=false2026.01 | 60.1 | 76.5 | 58.2 | 64.9 | |
| Qwen2.5-InstructParams=7B, Label=false, Knowledge=false2026.01 | 59.8 | 86.6 | 55.6 | 67.3 | |
| MediTronParams=70B, Label=true, Knowledge=true2026.01 | 59.8 | 71.5 | 53.3 | 61.5 | |
| Baseline-w/-labelParams=3B, Label=true, Knowledge=false2026.01 | 59.1 | 77.3 | 57.6 | 64.7 | |
| Baseline-w/o-labelParams=3B, Label=false, Knowledge=false2026.01 | 58.4 | 73.5 | 56.5 | 62.8 | |
| InstructDiffBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 54.67 | 64.48 | 50.11 | 56.42 | |
| EntropyMaxBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 52.4 | 66.52 | 52.16 | 57.03 | |
| ZIPBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 51.53 | 62.87 | 48.84 | 54.41 | |
| Inst/RespMaxBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 50.98 | 58.71 | 43.92 | 51.2 | |
| DGRC (Ours)Params=1.5B, Label=false, Knowledge=false2026.01 | 50.8 | 72.2 | 51.7 | 58.3 | |
| ALLBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 49.96 | 61.04 | 48.41 | 53.14 | |
| Baseline-w/-labelParams=1.5B, Label=true, Knowledge=false2026.01 | 49.8 | 72.2 | 49.2 | 57.1 | |
| RLAIF (GRPO)Params=1.5B, Label=false, Knowledge=false2026.01 | 49.5 | 67.5 | 50.2 | 55.7 | |
| Inst LenMaxBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 49.25 | 57.1 | 45.52 | 50.62 | |
| Baseline-w/o-labelParams=1.5B, Label=false, Knowledge=false2026.01 | 48.3 | 65 | 48.9 | 54.1 | |
| PPLMinBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 48.23 | 57.69 | 43.51 | 49.81 | |
| PPLMaxBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 47.76 | 58.38 | 45.37 | 50.5 | |
| EntropyMidBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 46.43 | 53.3 | 42.19 | 47.31 | |
| SelectITBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 46.11 | 46.84 | 35.93 | 42.96 | |
| RandomBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 45.95 | 54.22 | 40.11 | 46.76 | |
| PPLMidBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 43.75 | 48.45 | 35.09 | 42.43 | |
| Base ModelBackbone=LLaMA3-8B, Selection Ratio=0%2026.01 | 43.6 | 51.04 | 48.98 | 47.87 | |
| IFDBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 43.13 | 48.27 | 42.36 | 44.59 | |
| Resp LenMaxBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 42.11 | 41.99 | 32.44 | 38.85 | |
| SuperfilteringBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 41.63 | 41.15 | 32.08 | 38.29 | |
| Qwen2.5-InstructParams=3B, Label=false, Knowledge=false2026.01 | 41.6 | 63.7 | 38.4 | 47.9 | |
| Qwen2.5-InstructParams=1.5B, Label=false, Knowledge=false2026.01 | 40.1 | 54.8 | 37.1 | 44 | |
| MegaScienceParams=3B, Label=false, Knowledge=true2026.01 | 40 | 73.3 | 44.3 | 52.5 | |
| EntropyMinBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 36.76 | 42.24 | 34.23 | 37.74 | |
| Inst/RespMinBackbone=LLaMA3-8B, Selection Ratio=10%2026.01 | 35.51 | 44.65 | 40.16 | 40.11 | |
| MegaScienceParams=1.5B, Label=false, Knowledge=true2026.01 | 31.2 | 61.2 | 36.5 | 43 |