Medical Question Answering on MedQA (Accuracy)
86.5AccuracyMed-PaLM 2
Evaluation results
| Method | Links | |
|---|---|---|
| Med-PaLM 2RAG Protocol=No-RAG baselines2026.04 | 86.5 | |
| GPT-4-baseRAG Protocol=No-RAG baselines, Few-shot settings=5-shot2026.04 | 86.1 | |
| Hybrid-IRBackbone=GPT-4o-mini2026.06 | 82.9 | |
| MedRAG + GPT-4RAG Protocol=Text-only RAG2026.04 | 82.8 | |
| i-MedRAGBackbone=GPT-4o-mini2026.06 | 80.5 | |
| GPT-4o-miniRAG=None2026.06 | 80.3 | |
| MEDVRAG (full system)Memory Bank Usage=memory bank (full system)2026.04 | 79.4 | |
| RecursiveMASrecursion round=32026.04 | 79.3 | |
| GFM-RAGBackbone=GPT-4o-mini2026.06 | 78.5 | |
| MEDVRAG (+ iterative reasoning)Reasoning Rounds=iterative, Memory Bank Usage=no memory2026.04 | 77.6 | |
| TextGradrecursion round=32026.04 | 77.2 | |
| Single Agenttraining protocol=Full-SFT2026.04 | 77 | |
| Recursive-TextMASrecursion round=32026.04 | 77 | |
| Single Agenttraining protocol=LoRA2026.04 | 76.1 | |
| MEDVRAG (single-round)Reasoning Rounds=single-round, Data format=page images2026.04 | 75.8 | |
| MedRAGBackbone=GPT-4o-mini2026.06 | 75.5 | |
| HippoRAGBackbone=GPT-4o-mini2026.06 | 75.1 | |
| MEDVRAG (text-only retrieval)Retrieval Type=text-only2026.04 | 74.8 | |
| MedCPTBackbone=GPT-4o-mini, k=12026.06 | 74.3 | |
| SFT-OracleBackbone=Qwen2.5-7B, |Train|=8.1K2026.05 | 73.99 | |
| HuatuoGPT-o1-8BRAG Protocol=No-RAG baselines2026.04 | 72.6 | |
| Yi-34B (OpenMedLM)RAG Protocol=No-RAG baselines2026.04 | 72.6 | |
| Hybrid-IRBackbone=Llama-3-8B-Instruct2026.06 | 72.2 | |
| i-MedRAGBackbone=Llama-3-8B-Instruct2026.06 | 71.8 | |
| Qwen3-30BRAG Protocol=No-RAG baselines2026.04 | 71.5 | |
| MedGraphRAGBackbone=GPT-4o-mini2026.06 | 71.5 | |
| MEDVRAG (no RAG)RAG Protocol=No RAG2026.04 | 71.4 | |
| SYNDATAModel=Qwen3-4B, Data Size=40002026.05 | 70.62 | |
| HSIR-DPOBackbone=Qwen2.5-7B, Iteration=3, |Train|=19.6K2026.05 | 70.46 | |
| Meditron-70BRAG Protocol=No-RAG baselines2026.04 | 70 | |
| HSIR-SFTBackbone=Qwen2.5-7B, Iteration=3, |Train|=38.8K2026.05 | 68.74 | |
| HSIR-DPOBackbone=Qwen2.5-7B, Iteration=2, |Train|=23.6K2026.05 | 68.58 | |
| SYNDATAModel=Qwen2.5-7B, Data Size=40002026.05 | 68.12 | |
| H-GRPOBackbone=Qwen2.5-7B2026.05 | 68.03 | |
| HSIR-SFTBackbone=Qwen2.5-7B, Iteration=2, |Train|=36.6K2026.05 | 67.87 | |
| HSIR-SFTBackbone=Qwen2.5-7B, Iteration=1, |Train|=33.6K2026.05 | 67.32 | |
| HSIR-DPOBackbone=Qwen2.5-7B, Iteration=1, |Train|=17.9K2026.05 | 67.32 | |
| AR-OPD2026.06 | 66.7 | |
| MedRAG + GPT-3.5RAG Protocol=Text-only RAG2026.04 | 66.6 | |
| GRPOBackbone=Qwen2.5-7B2026.05 | 66.46 | |
| Long2ShortBackbone=Qwen2.5-7B2026.05 | 66.14 | |
| SYNDATAModel=Qwen3-4B, Data Size=20002026.05 | 66.13 | |
| SFT2026.06 | 66 | |
| ReSTEMBackbone=Qwen2.5-7B, Iteration=2, |Train|=51.0K2026.05 | 65.28 | |
| ReSTEMBackbone=Qwen2.5-7B, Iteration=3, |Train|=53.6K2026.05 | 65.28 | |
| Full OPD2026.06 | 65.2 | |
| RFTBackbone=Qwen2.5-7B, |Train|=132.2K2026.05 | 64.89 | |
| SYNDATAModel=Qwen3-4B, Data Size=10002026.05 | 64.7 | |
| SYNDATAModel=Qwen2.5-7B, Data Size=20002026.05 | 64.57 | |
| Partial OPD2026.06 | 64.3 | |
| IRPOBackbone=Qwen2.5-7B, Iteration=1, |Train|=22.7K2026.05 | 64.26 | |
| ReSTEMBackbone=Qwen2.5-7B, Iteration=1, |Train|=44.7K2026.05 | 64.18 | |
| CosFnBackbone=Qwen2.5-7B2026.05 | 63.71 | |
| Qwen3-8BRAG Protocol=No-RAG baselines2026.04 | 63.5 | |
| IRPOBackbone=Qwen2.5-7B, Iteration=2, |Train|=27.5K2026.05 | 63.24 | |
| SFT-InitialBackbone=Qwen2.5-7B, |Train|=1.0K2026.05 | 62.45 | |
| ReGenesisBackbone=Qwen2.5-7B, |Train|=127.2K2026.05 | 62.22 | |
| STaRBackbone=Qwen2.5-7B, Iteration=3, |Train|=6.4K2026.05 | 61.59 | |
| STaRBackbone=Qwen2.5-7B, Iteration=1, |Train|=5.7K2026.05 | 61.19 | |
| STaRBackbone=Qwen2.5-7B, Iteration=2, |Train|=6.1K2026.05 | 61.04 | |
| HSIR-DPOBackbone=Qwen2.5-3B, Iteration=3, |Train|=19.6K2026.05 | 60.64 | |
| IRPOBackbone=Qwen2.5-7B, Iteration=3, |Train|=20.5K2026.05 | 60.57 | |
| GFM-RAGBackbone=Llama-3-8B-Instruct2026.06 | 60.3 | |
| GPT-3.5RAG Protocol=No-RAG baselines2026.04 | 60.2 | |
| HSIR-DPOBackbone=Qwen2.5-3B, Iteration=2, |Train|=23.6K2026.05 | 59.15 | |
| SYNDATAModel=Qwen2.5-7B, Data Size=10002026.05 | 59.02 | |
| H-GRPOBackbone=Qwen2.5-3B2026.05 | 58.98 | |
| Base2026.06 | 58.7 | |
| SFT-OracleBackbone=Qwen2.5-3B, |Train|=8.1K2026.05 | 58.68 | |
| Long2ShortBackbone=Qwen2.5-3B2026.05 | 58.21 | |
| HippoRAGBackbone=Llama-3-8B-Instruct2026.06 | 57.9 | |
| MedGraphRAGBackbone=Llama-3-8B-Instruct2026.06 | 57.8 | |
| Llama-3-8B-InstructRAG=None2026.06 | 57.7 | |
| HSIR-SFTBackbone=Qwen2.5-3B, Iteration=3, |Train|=38.8K2026.05 | 57.58 | |
| Base ModelModel=Qwen3-4B, Data Size=-2026.05 | 57.52 | |
| Mixture-of-Agents (MoA)recursion round=32026.04 | 57.5 | |
| GRPOBackbone=Qwen2.5-3B2026.05 | 57.03 | |
| POPModel=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.04 | 56.54 | |
| ReSTEMBackbone=Qwen2.5-3B, Iteration=3, |Train|=53.6K2026.05 | 56.48 | |
| LoopLMrecursion round=32026.04 | 56.4 | |
| MedRAGBackbone=Llama-3-8B-Instruct2026.06 | 56.4 | |
| HSIR-SFTBackbone=Qwen2.5-3B, Iteration=2, |Train|=36.6K2026.05 | 56.32 | |
| ReSTEMBackbone=Qwen2.5-3B, Iteration=2, |Train|=51.0K2026.05 | 56.25 | |
| BaseModel=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.04 | 55.94 | |
| HSIR-DPOBackbone=Qwen2.5-3B, Iteration=1, |Train|=17.9K2026.05 | 55.77 | |
| HSIR-SFTBackbone=Qwen2.5-3B, Iteration=1, |Train|=33.6K2026.05 | 55.7 | |
| MedCPTBackbone=Llama-3-8B-Instruct, k=12026.06 | 55.3 | |
| CosFnBackbone=Qwen2.5-3B2026.05 | 55.22 | |
| ReSTEMBackbone=Qwen2.5-3B, Iteration=1, |Train|=44.7K2026.05 | 55.22 | |
| RFTBackbone=Qwen2.5-3B, |Train|=132.2K2026.05 | 54.6 | |
| POPModel=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.04 | 53.86 | |
| PRL2025.05 | 53.34 | |
| Train on DModel=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.04 | 53.12 | |
| BaseModel=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.04 | 52.95 | |
| GRACE2025.05 | 52.26 | |
| ReConcileBase Model=Gemma-3-4B2025.08 | 52 | |
| GA2025.05 | 51.95 | |
| PromptAgent2025.05 | 51.87 | |
| PromptWizard2025.05 | 51.84 | |
| DE2025.05 | 51.76 |