Medical Question Answering on MedExpQA
86.19Overall AccuracyGemini-2.5-Flash
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-2.5-FlashCategory=Proprietary2026.01 | 86.19 | 85.92 | 88 | 88.96 | 88.64 | 87.68 | 86.08 | 88.48 | 88.48 | 88.96 | 77.76 | 83.04 | 82.24 | — | |
| o4-miniCategory=Proprietary2026.01 | 82.99 | 84.16 | 88.48 | 90.08 | 88.64 | 87.04 | 85.44 | 88.8 | 88.64 | 87.84 | 44.96 | 79.04 | 82.72 | — | |
| GPT-5Category=Proprietary2026.01 | 82.63 | 84.96 | 87.2 | 87.52 | 87.68 | 85.92 | 86.4 | 86.72 | 86.24 | 86.72 | 56.96 | 73.28 | 81.92 | — | |
| DeepSeek-R1Category=Open-Weight2026.01 | 82.25 | 84.96 | 87.68 | 88.32 | 87.84 | 88 | 78.72 | 87.52 | 88.64 | 84.16 | 64.8 | 71.52 | 74.88 | — | |
| gpt-oss-120BCategory=Open-Weight2026.01 | 81.91 | 86.4 | 88.16 | 88.64 | 90.24 | 87.04 | 84.96 | 88 | 87.68 | 78.08 | 55.2 | 74.56 | 73.92 | — | |
| Claude-4.0-SonnetCategory=Proprietary2026.01 | 80.65 | 83.04 | 87.04 | 88.16 | 86.88 | 83.68 | 87.04 | 88.16 | 88.32 | 80 | 56.32 | 68.96 | 70.24 | — | |
| LLaMA-4-MaverickCategory=Open-Weight2026.01 | 80.17 | 83.36 | 86.24 | 88.16 | 86.88 | 83.36 | 82.88 | 88.16 | 87.68 | 80.48 | 54.72 | 67.68 | 72.48 | — | |
| GPT-4.1Category=Proprietary2026.01 | 79.84 | 84.32 | 84 | 87.36 | 84.8 | 86.72 | 81.12 | 88.16 | 85.28 | 84.96 | 52.64 | 64.8 | 73.92 | — | |
| MA-RAG-extBackbone=Qwen3-8B, Ranking Agent=extrinsic verification2026.02 | 78.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4oCategory=Proprietary2026.01 | 76.85 | 83.2 | 85.44 | 87.04 | 87.52 | 82.88 | 82.4 | 85.76 | 85.44 | 82.08 | 36.16 | 52.48 | 71.84 | — | |
| GPT-5-miniCategory=Proprietary2026.01 | 76.11 | 80 | 84 | 85.6 | 85.76 | 82.88 | 80.32 | 85.44 | 84.48 | 80 | 29.92 | 61.6 | 73.28 | — | |
| DeepSeek-V3Category=Open-Weight2026.01 | 75.96 | 82.56 | 84.48 | 86.08 | 82.4 | 84.96 | 80.96 | 84.48 | 87.36 | 76.96 | 47.36 | 54.56 | 59.36 | — | |
| TC-RAGBackbone=Qwen3-8B2026.02 | 75.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4.1-miniCategory=Proprietary2026.01 | 74.79 | 80.32 | 86.08 | 85.92 | 86.4 | 82.08 | 80.48 | 85.12 | 84.64 | 75.84 | 25.44 | 56.64 | 68.48 | — | |
| Multi-RefineBackbone=Qwen3-8B2026.02 | 73.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SR-RAGBackbone=Qwen3-8B2026.02 | 72.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MA-RAG-intBackbone=Qwen3-8B, Ranking Agent=intrinsic uncertainty2026.02 | 72.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| gpt-oss-20BCategory=Open-Weight2026.01 | 72.31 | 81.28 | 76.96 | 82.08 | 84.32 | 80.8 | 76.16 | 82.4 | 81.12 | 68.64 | 28.48 | 58.56 | 66.88 | — | |
| CoTBackbone=Qwen3-8B2026.02 | 72 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLAREBackbone=Qwen3-8B2026.02 | 71.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SCBackbone=Qwen3-8B2026.02 | 70.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FL-RAGBackbone=Qwen3-8B2026.02 | 69.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA-4-ScoutCategory=Open-Weight2026.01 | 69.04 | 75.2 | 83.2 | 79.2 | 81.28 | 72.16 | 72.8 | 78.56 | 77.12 | 68 | 44.32 | 38.24 | 58.4 | — | |
| MedGemma-27BCategory=Open-Weight2026.01 | 69.03 | 75.04 | 82.56 | 80 | 80.8 | 74.72 | 74.08 | 80.16 | 80.16 | 76 | 17.6 | 45.6 | 61.6 | — | |
| Phi-4-ReasoningCategory=Open-Weight2026.01 | 68.04 | 79.52 | 85.6 | 85.44 | 84.16 | 83.52 | 75.52 | 74.08 | 84 | 72 | 17.76 | 44.16 | 30.72 | — | |
| HuatuoGPT-o1-72BCategory=Open-Weight2026.01 | 67.55 | 79.52 | 80.32 | 84.16 | 83.04 | 80.8 | 76.48 | 85.6 | 84 | 61.92 | 39.68 | 36 | 19.04 | — | |
| Qwen3-8BBackbone=Qwen3-8B2026.02 | 67.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.1-8BBackbone=Llama-3.1-8B2026.02 | 67.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FS-RAGBackbone=Qwen3-8B2026.02 | 67.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HuatuoGPT-o1-70BCategory=Open-Weight2026.01 | 66.71 | 62.08 | 80.16 | 78.88 | 79.84 | 64 | 72.48 | 79.36 | 80.8 | 70.88 | 33.92 | 43.84 | 54.24 | — | |
| LLaMA-3.3-70BCategory=Open-Weight2026.01 | 66.51 | 66.08 | 82.4 | 81.44 | 80.48 | 58.72 | 66.88 | 78.56 | 81.44 | 67.52 | 43.04 | 42.72 | 48.8 | — | |
| LLaMA-3.1-70BCategory=Open-Weight2026.01 | 66.48 | 73.6 | 80.64 | 78.88 | 78.08 | 72.48 | 66.56 | 76.48 | 77.28 | 67.84 | 43.68 | 38.56 | 43.68 | — | |
| UltraMedical-3.1-8BBackbone=UltraMedical-3.1-8B2026.02 | 66.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Claude-3.5-HaikuCategory=Proprietary2026.01 | 64.63 | 72.64 | 78.4 | 80 | 75.36 | 74.4 | 72 | 80.8 | 80 | 49.76 | 33.76 | 43.2 | 35.2 | — | |
| Qwen2.5-72BCategory=Open-Weight2026.01 | 64.59 | 76.48 | 78.88 | 79.68 | 77.6 | 75.52 | 70.08 | 80.48 | 79.2 | 50.24 | 42.24 | 28.96 | 35.68 | — | |
| QwQ-32BCategory=Open-Weight2026.01 | 63.84 | 81.92 | 83.68 | 84.48 | 82.56 | 79.04 | 68.96 | 84.16 | 83.68 | 64 | 17.92 | 15.84 | 19.84 | — | |
| HuatuoGPT-o1-8BBackbone=HuatuoGPT-o1-8B2026.02 | 63.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4o-miniCategory=Proprietary2026.01 | 62.75 | 68.16 | 75.52 | 78.88 | 73.28 | 70.24 | 62.88 | 76.32 | 74.72 | 61.76 | 27.84 | 38.08 | 45.28 | — | |
| Gemma-3-27BCategory=Open-Weight2026.01 | 61.79 | 67.84 | 74.72 | 71.36 | 74.24 | 66.88 | 64.16 | 72.32 | 72.96 | 62.4 | 16.16 | 41.76 | 56.64 | — | |
| Qwen3-14B-thinkingCategory=Open-Weight2026.01 | 61.08 | 78.72 | 79.68 | 79.52 | 81.76 | 80.64 | 73.6 | 80.16 | 77.92 | 57.6 | 18.08 | 11.68 | 13.6 | — | |
| GPT-4.1-nanoCategory=Proprietary2026.01 | 59.6 | 66.08 | 78.08 | 73.44 | 72.96 | 61.92 | 57.12 | 74.56 | 71.68 | 52 | 29.44 | 35.2 | 42.72 | — | |
| Baichuan-M2-32BCategory=Open-Weight2026.01 | 59.33 | 79.36 | 83.04 | 76.8 | 79.52 | 77.12 | 67.52 | 80 | 74.56 | 26.56 | 20.64 | 23.04 | 23.84 | — | |
| Phi-4Category=Open-Weight2026.01 | 54.57 | 59.04 | 79.2 | 69.6 | 74.56 | 62.4 | 52 | 71.36 | 69.44 | 41.44 | 19.04 | 30.56 | 26.24 | — | |
| Qwen3-14BCategory=Open-Weight2026.01 | 54.17 | 73.12 | 74.08 | 73.28 | 72.8 | 67.84 | 58.88 | 75.52 | 73.44 | 38.24 | 13.12 | 13.92 | 15.84 | — | |
| Qwen3-8B-thinkingCategory=Open-Weight2026.01 | 54.11 | 73.28 | 76.64 | 73.28 | 74.72 | 70.88 | 67.36 | 74.24 | 74.72 | 26.08 | 14.72 | 13.28 | 10.08 | — | |
| GPT-5-nanoCategory=Proprietary2026.01 | 53.8 | 51.84 | 64 | 60.96 | 65.92 | 68 | 52.64 | 64.48 | 63.68 | 57.12 | 17.76 | 39.2 | 40 | — | |
| Qwen2.5-14BCategory=Open-Weight2026.01 | 52.92 | 67.52 | 73.28 | 60 | 69.28 | 62.08 | 55.36 | 68 | 67.36 | 31.84 | 20.32 | 28.16 | 31.84 | — | |
| Gemma-3-12BCategory=Open-Weight2026.01 | 52.84 | 61.92 | 68 | 64.8 | 65.44 | 55.36 | 54.72 | 66.24 | 65.76 | 53.6 | 10.08 | 27.04 | 41.12 | — | |
| Mistral-Small-3.1-24BCategory=Open-Weight2026.01 | 52.28 | 68.96 | 78.24 | 71.36 | 76.16 | 66.88 | 54.24 | 72.96 | 72.48 | 30.72 | 10.88 | 12.48 | 12 | — | |
| DeepSeek-R1-Qwen3-8BCategory=Open-Weight2026.01 | 50.08 | 70.24 | 73.76 | 66.56 | 64 | 63.84 | 63.68 | 66.56 | 66.24 | 28.32 | 11.68 | 12.96 | 13.12 | — | |
| Qwen3-4B-thinkingCategory=Open-Weight2026.01 | 48.37 | 66.08 | 72 | 66.24 | 66.08 | 62.4 | 56 | 66.56 | 63.52 | 21.12 | 18.08 | 13.6 | 8.8 | — | |
| Qwen3-8BCategory=Open-Weight2026.01 | 45.81 | 63.68 | 70.24 | 58.88 | 63.36 | 58.56 | 53.44 | 61.44 | 60.32 | 16.96 | 11.52 | 22.24 | 9.12 | — | |
| MedGemma-4BCategory=Open-Weight2026.01 | 44.77 | 48 | 66.56 | 55.68 | 60.64 | 45.12 | 41.44 | 54.56 | 54.24 | 41.6 | 17.12 | 21.76 | 30.56 | — | |
| Qwen2.5-7BCategory=Open-Weight2026.01 | 43.75 | 57.76 | 64.8 | 59.52 | 52.32 | 52.96 | 48.16 | 55.2 | 54.08 | 31.2 | 14.24 | 20.16 | 14.56 | — | |
| Qwen3-4BCategory=Open-Weight2026.01 | 43.23 | 61.92 | 66.88 | 56.16 | 61.28 | 53.12 | 47.36 | 56 | 56.96 | 17.28 | 15.52 | 16.16 | 10.08 | — | |
| HuatuoGPT-o1-8BCategory=Open-Weight2026.01 | 42.96 | 55.52 | 63.84 | 60.32 | 57.28 | 47.52 | 42.56 | 57.76 | 58.88 | 40.64 | 11.2 | 11.68 | 8.32 | — | |
| HuatuoGPT-o1-7BCategory=Open-Weight2026.01 | 42.48 | 59.2 | 58.08 | 62.24 | 63.52 | 56.48 | 54.72 | 57.92 | 57.76 | 10.72 | 11.68 | 9.76 | 7.68 | — | |
| LLaMA-3.1-8BCategory=Open-Weight2026.01 | 38.96 | 44.96 | 68.64 | 52.32 | 50.72 | 41.6 | 30.24 | 55.04 | 47.84 | 30.56 | 21.92 | 13.12 | 10.56 | — | |
| Bio-Medical-LLaMA-3-8BCategory=Open-Weight2026.01 | 38.75 | 43.84 | 62.88 | 46.4 | 48.16 | 41.76 | 31.52 | 40 | 42.88 | 30.88 | 24.8 | 27.36 | 24.48 | — | |
| OpenBioLLM-70BCategory=Open-Weight2026.01 | 35.97 | 25.76 | 70.72 | 63.68 | 40.8 | 26.72 | 21.44 | 47.2 | 58.56 | 29.12 | 10.72 | 14.24 | 22.72 | — | |
| Gemma-3-4BCategory=Open-Weight2026.01 | 35.69 | 42.88 | 52.96 | 47.68 | 45.76 | 42.08 | 33.6 | 44.96 | 44.96 | 35.52 | 10.72 | 15.36 | 11.84 | — | |
| Qwen3-1.7BCategory=Open-Weight2026.01 | 35.32 | 49.28 | 51.36 | 44 | 41.28 | 35.52 | 33.44 | 41.44 | 41.92 | 15.84 | 22.4 | 25.76 | 21.6 | — | |
| Qwen2.5-3BCategory=Open-Weight2026.01 | 34.53 | 50.4 | 49.28 | 46.88 | 43.68 | 44 | 36.16 | 43.84 | 44.8 | 14.88 | 13.12 | 14.88 | 12.48 | — | |
| LLaMA-3.2-3BCategory=Open-Weight2026.01 | 31.93 | 39.84 | 58.88 | 45.6 | 39.52 | 37.44 | 22.88 | 43.84 | 27.84 | 25.76 | 12.64 | 16.32 | 12.64 | — | |
| MediPhiCategory=Open-Weight2026.01 | 31.47 | 28.96 | 53.28 | 44.16 | 50.56 | 24.16 | 29.12 | 38.72 | 39.52 | 13.44 | 17.76 | 17.28 | 20.64 | — | |
| Phi-4-mini-ReasoningCategory=Open-Weight2026.01 | 31.32 | 28.96 | 61.12 | 46.24 | 49.6 | 24.8 | 15.68 | 35.2 | 39.04 | 24 | 18.4 | 16.32 | 16.48 | — | |
| Phi-4-miniCategory=Open-Weight2026.01 | 31.21 | 34.56 | 58.08 | 36.8 | 40.8 | 35.84 | 28 | 35.68 | 34.08 | 22.88 | 14.56 | 16.64 | 16.64 | — | |
| MedReason-8BCategory=Open-Weight2026.01 | 23.53 | 46.56 | 54.56 | 16.96 | 16.64 | 39.84 | 32 | 18.4 | 21.76 | 9.76 | 9.6 | 9.6 | 6.72 | — | |
| Mistral-7B-v0.3Category=Open-Weight2026.01 | 21.8 | 28.48 | 32.64 | 15.52 | 39.04 | 19.36 | 19.52 | 19.2 | 30.4 | 16.96 | 18.56 | 11.2 | 10.72 | — | |
| OpenBioLLM-8BCategory=Open-Weight2026.01 | 16.21 | 17.44 | 30.24 | 14.56 | 24.8 | 9.28 | 10.4 | 24.8 | 21.6 | 12 | 6.56 | 9.12 | 13.76 | — | |
| HuatuoGPTfamily=HuatuoGPT, size=8B2025.12 | — | — | 67 | — | — | — | — | — | 56.3 | — | — | — | — | 56.4 | |
| m1family=m1, size=7B2025.12 | — | — | 62.2 | — | — | — | — | — | 48.1 | — | — | — | — | 48.6 | |
| MedReasonfamily=MedReason, size=8B2025.12 | — | — | 64.3 | — | — | — | — | — | 58.4 | — | — | — | — | 55.1 | |
| Ourfamily=Our, size=8B2025.12 | — | — | 69.7 | — | — | — | — | — | 69.7 | — | — | — | — | 62.2 | |
| Ourfamily=Our, size=8B, shots=2-shot2025.12 | — | — | 68.1 | — | — | — | — | — | 65 | — | — | — | — | 64 |