Question Answering on PubMedQA (test)
82.4AccuracyAccurateRAG
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| AccurateRAGLLM Backbone=Llama-3-8B, Fine-tuning=true2025.10 | 82.4 | — | — | — | — | — | — | — | |
| Med-PaLM 2Prompting strategy=best2023.05 | 81.8 | — | — | — | — | — | — | — | |
| DEEPMED-14B-RLModel Group=DeepResearch Models, Training Stage=RL2026.01 | 81.8 | — | — | — | — | — | — | — | |
| GPT-4Date=20232022.07 | 81.2 | — | — | — | — | — | — | — | |
| Med-PaLM 2Study=Singhal et al. [6]2025.12 | 81 | — | — | — | — | — | — | — | |
| GPT-4Model=GPT-4, Setting=Few-Shot2025.12 | 81 | — | — | — | — | — | — | — | |
| AlphaMed-70BModel Group=Medical Reasoning Models2026.01 | 80.9 | — | — | — | — | — | — | — | |
| HuatuoGPT-o1-70BModel Group=Medical Reasoning Models2026.01 | 80.6 | — | — | — | — | — | — | — | |
| GPT-4-baseFew-shot evaluation protocol=5-shot2023.05 | 80.4 | — | — | — | — | — | — | — | |
| DEEPMED-14B-SFTModel Group=DeepResearch Models, Training Stage=SFT2026.01 | 80.4 | — | — | — | — | — | — | — | |
| BaiChuan-M2-32BModel Group=Medical Reasoning Models2026.01 | 79.2 | — | — | — | — | — | — | — | |
| Flan-PaLMPrompting strategy=best2023.05 | 79 | — | — | — | — | — | — | — | |
| QuarkMed-32BModel Group=Medical Reasoning Models2026.01 | 79 | — | — | — | — | — | — | — | |
| PubMed ReasonerBackbone Model=GPT2026.03 | 78.32 | — | — | — | — | — | — | — | |
| Codex 5-shot CoTDate=2022, Few-shot shots=5, Chain-of-Thought (CoT)=true, Ensemble samples (k)=1002022.07 | 78.2 | — | — | — | — | — | — | — | |
| BioGPTDate=2022, Evaluation Protocol=Finetuned SOTA2022.07 | 78.2 | — | — | — | — | — | — | — | |
| BioGPTStudy=Luo et al. [8]2025.12 | 78.2 | — | — | — | — | — | — | — | |
| GPT-4Model=GPT-4, Setting=Zero-Shot2025.12 | 78.2 | — | — | — | — | — | — | — | |
| Human (expert score)2022.07 | 78 | — | — | — | — | — | — | — | |
| Human performanceRetrieval/Agent Strategy=Without retrieval2026.03 | 78 | — | — | — | — | — | — | — | |
| MeissaFramework=Framework III, Inference Mode=Agent-based, Agent System=MDAgents2026.03 | 77.9 | — | — | — | — | — | — | — | |
| M1-1K-32BModel Group=Medical Reasoning Models2026.01 | 77.6 | — | — | — | — | — | — | — | |
| MedResaon-8BModel Group=Medical Reasoning Models2026.01 | 77.6 | — | — | — | — | — | — | — | |
| M1-1K-7BModel Group=Medical Reasoning Models2026.01 | 77.5 | — | — | — | — | — | — | — | |
| MedPalm v2Date=20232022.07 | 77.4 | — | — | — | — | — | — | — | |
| PubMed ReasonerBackbone Model=Gemini2026.03 | 77.26 | — | — | — | — | — | — | — | |
| Self-reflectionBackbone Model=GPT2026.03 | 77.12 | — | — | — | — | — | — | — | |
| Self-reflectionBackbone Model=Gemini2026.03 | 77.08 | — | — | — | — | — | — | — | |
| Qwen3-VL-4BFramework=Framework III, Inference Mode=Direct Inference, Training Protocol=SFT2026.03 | 76.8 | — | — | — | — | — | — | — | |
| Kimi-K2-Thinking-1TBModel Group=Awesome General Models2026.01 | 76.6 | — | — | — | — | — | — | — | |
| PubMed ReasonerBackbone=Qwen2.5-72B2026.03 | 76.42 | — | — | — | — | — | — | — | |
| Self-reflection agentBackbone=Qwen2.5-72B2026.03 | 76.22 | — | — | — | — | — | — | — | |
| Tongyi-DeepResearch-30BA3BModel Group=DeepResearch Models2026.01 | 76.2 | — | — | — | — | — | — | — | |
| Gemini2.5-ProModel Group=Awesome General Models2026.01 | 76 | — | — | — | — | — | — | — | |
| Qwen3-30BA3B-ThinkingModel Group=Awesome General Models2026.01 | 76 | — | — | — | — | — | — | — | |
| Deepseek-v3.2-685BModel Group=Awesome General Models2026.01 | 75.8 | — | — | — | — | — | — | — | |
| Gemini-2.5 ProRetrieval/Agent Strategy=Without retrieval2026.03 | 75.64 | — | — | — | — | — | — | — | |
| Llama 3.1 8B InstructModel=Llama 3.1, Parameters=8B, Type=Instruct2026.04 | 75.6 | — | — | 54.06 | — | — | — | — | |
| Qwen3-14BModel Group=Awesome General Models2026.01 | 75.4 | — | — | — | — | — | — | — | |
| GPT-4Few-shot evaluation protocol=5-shot2023.05 | 75.2 | — | — | — | — | — | — | — | |
| GPT4oRetrieval/Agent Strategy=Without retrieval, Source=leaderboard2026.03 | 75.2 | — | — | — | — | — | — | — | |
| Med-PaLM 2Prompting strategy=Ensemble Refinement (ER)2023.05 | 75 | — | — | — | — | — | — | — | |
| GPT-4VFramework=Framework III, Inference Mode=Agent-based, Agent System=MDAgents2026.03 | 75 | — | — | — | — | — | — | — | |
| Without retrievalBackbone=Qwen2.5-72B2026.03 | 74.89 | — | — | — | — | — | — | — | |
| AccurateRAGLLM Backbone=Llama-2-7B, Fine-tuning=true2025.10 | 74.6 | — | — | — | — | — | — | — | |
| DRAGONBackbone=BioLinkBERT-Large2022.10 | 73.4 | — | — | — | — | — | — | — | |
| RAFTLLM Backbone=Llama-2-7B, Fine-tuning=true, Chain-of-Thought (CoT)=GPT-4 CoT2025.10 | 73.3 | — | — | — | — | — | — | — | |
| RAG methodBackbone Model=GPT2026.03 | 73.28 | — | — | — | — | — | — | — | |
| GPT-3.5 Turbo 1106shot=3-shot2024.02 | 72.66 | — | — | — | — | — | — | — | |
| BioLinkBERT + GreaseLMBackbone=BioLinkBERT-Large, KG Module=GreaseLM2022.10 | 72.4 | — | — | — | — | — | — | — | |
| RAG methodBackbone Model=Gemini2026.03 | 72.3 | — | — | — | — | — | — | — | |
| BioLinkBERTBackbone=BioLinkBERT-Large2022.10 | 72.2 | — | — | — | — | — | — | — | |
| Mistral 7B Instructshot=3-shot2024.02 | 72.2 | — | — | — | — | — | — | — | |
| BioLinkBERT + QAGNNBackbone=BioLinkBERT-Large, KG Module=QAGNN2022.10 | 72.1 | — | — | — | — | — | — | — | |
| Gemini-3-flashFramework=Framework III, Inference Mode=Agent-based, Agent System=MDAgents2026.03 | 71.9 | — | — | — | — | — | — | — | |
| LLaMA 2 + RAGStudy=This Work, Model size=13B, Setting=Fine-Tuned + RAG2025.12 | 71.8 | — | — | — | — | — | — | — | |
| LLaMA 2 + RAGModel=LLaMA 2, Setting=Fine-Tuned + RAG2025.12 | 71.8 | — | — | — | — | — | — | — | |
| OracleModel Size=Large, Expertise Distribution=Dist. 32026.04 | 71.25 | — | — | — | — | — | — | — | |
| OracleModel Size=Large, Noise Type=Uniform2026.04 | 71.25 | — | — | — | — | — | — | — | |
| Self-reflection agentBackbone=Qwen2.5-7B2026.03 | 71.12 | — | — | — | — | — | — | — | |
| PubMed ReasonerBackbone=Qwen2.5-7B2026.03 | 71.08 | — | — | — | — | — | — | — | |
| Without retrievalBackbone=Qwen2.5-7B2026.03 | 70.2 | — | — | — | — | — | — | — | |
| BioMistral 7B DAREshot=3-shot2024.02 | 70 | — | — | — | — | — | — | — | |
| Falcon + RAGModel=Falcon, Setting=Fine-Tuned + RAG2025.12 | 68.9 | — | — | — | — | — | — | — | |
| RAFTLLM Backbone=Llama-2-7B, Fine-tuning=true, Chain-of-Thought (CoT)=false2025.10 | 68.3 | — | — | — | — | — | — | — | |
| BioMistral 7B SLERPshot=3-shot2024.02 | 68.1 | — | — | — | — | — | — | — | |
| BioBERTStudy=Jin et al. [17]2025.12 | 68.1 | — | — | — | — | — | — | — | |
| Without retrievalBackbone=Qwen2.5-1.5B2026.03 | 68.08 | — | — | — | — | — | — | — | |
| BioBERT2026.04 | 68.08 | — | — | 52.72 | — | — | — | — | |
| Self-reflection agentBackbone=Qwen2.5-1.5B2026.03 | 68.06 | — | — | — | — | — | — | — | |
| BioMistral 7B Ensembleshot=3-shot2024.02 | 68 | — | — | — | — | — | — | — | |
| LoRA FT 2ep 10k contextFine-Tuning=LoRA, Epochs=2, Context Length=10k2026.04 | 68 | — | — | 46.69 | — | — | — | — | |
| PubMed ReasonerBackbone=Qwen2.5-1.5B2026.03 | 67.24 | — | — | — | — | — | — | — | |
| Gemini-3-flashFramework=Framework III, Inference Mode=Direct Inference2026.03 | 66.7 | — | — | — | — | — | — | — | |
| RankRAGLLM Backbone=Llama-3-8B, Fine-tuning=true2025.10 | 65 | — | — | — | — | — | — | — | |
| LLaMA 2Model=LLaMA 2, Setting=Fine-Tuned2025.12 | 64.3 | — | — | — | — | — | — | — | |
| SFT2026.03 | 64.2 | — | — | — | — | — | — | — | |
| NOISYModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. 32026.04 | 64.05 | — | — | — | — | — | — | — | |
| PPO2026.03 | 63.6 | — | — | — | — | — | — | — | |
| Full FT 5ep 16k contextFine-Tuning=Full, Epochs=5, Context Length=16k2026.04 | 62.8 | — | — | 38.69 | — | — | — | — | |
| Llama 3.2 3B InstructModel=Llama 3.2, Parameters=3B, Type=Instruct2026.04 | 62.4 | — | — | 40.93 | — | — | — | — | |
| TEPiterations (free phase)=20, iterations (nudged phase)=402026.01 | 62.02 | — | — | — | — | — | — | — | |
| GPT-4VFramework=Framework III, Inference Mode=Direct Inference2026.03 | 61.5 | — | — | — | — | — | — | — | |
| FalconModel=Falcon, Setting=Fine-Tuned2025.12 | 61.2 | — | — | — | — | — | — | — | |
| OracleModel Size=Base, Expertise Distribution=Dist. 32026.04 | 61 | — | — | — | — | — | — | — | |
| OracleModel Size=Base, Noise Type=Uniform2026.04 | 61 | — | — | — | — | — | — | — | |
| REALMModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. 32026.04 | 60.95 | — | — | — | — | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 60.55 | — | — | — | — | — | — | — | |
| DSPy2026.01 | 60.26 | — | — | — | — | — | — | — | |
| BioBERT2022.10 | 60.2 | — | — | — | — | — | — | — | |
| HBC2026.01 | 58.8 | — | — | — | — | — | — | — | |
| BioMedGPT-LM-7Bshot=3-shot2024.02 | 58.6 | — | — | — | — | — | — | — | |
| NOISYModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. 32026.04 | 58.45 | — | — | — | — | — | — | — | |
| OracleModel Size=Small, Expertise Distribution=Dist. 32026.04 | 58 | — | — | — | — | — | — | — | |
| OracleModel Size=Small, Noise Type=Uniform2026.04 | 58 | — | — | — | — | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 57.65 | — | — | — | — | — | — | — | |
| CoT2026.01 | 57.34 | — | — | — | — | — | — | — | |
| Qwen3-VL-4BFramework=Framework III, Inference Mode=Agent-based, Agent System=MDAgents2026.03 | 57.3 | — | — | — | — | — | — | — | |
| TextGradsummarization=false2026.01 | 56.96 | — | — | — | — | — | — | — | |
| RAAT + DTABase Model=Llama-2-7B, Alignment=Divide-Then-Align2025.05 | 56.6 | 59.1 | 56.2 | 57.6 | 52.1 | 57.5 | 54.5 | — |