Medical Question Answering on PubMedQA (Accuracy, F1, Avg. Accuracy)
82.8AccuracySafeLoRA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SafeLoRAModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 82.8 | — | — | |
| Med-PaLM 2RAG Protocol=No-RAG baselines2026.04 | 81.8 | — | — | |
| Meditron-70BRAG Protocol=No-RAG baselines2026.04 | 81.6 | — | — | |
| SafeMERGEModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 80.3 | — | — | |
| SafeInstructModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 80 | — | — | |
| Fine-tunedModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 79.6 | — | — | |
| SafeMERGEModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 79.5 | — | — | |
| SafeInstructModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 79.2 | — | — | |
| HuatuoGPT-o1-8BRAG Protocol=No-RAG baselines2026.04 | 79.2 | — | — | |
| SafeLoRAModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 79.1 | — | — | |
| SafeMERGEModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 79 | — | — | |
| Fine-tunedModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 78.8 | — | — | |
| Fine-tunedModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 78.8 | — | — | |
| SafeInstructModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 78.5 | — | — | |
| RESTA-InstructModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 78.5 | — | — | |
| MediHiveAgent Configuration=Multi-Agent2026.03 | 78.4 | 76.8 | 81.4 | |
| SafeLoRAModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 78.3 | — | — | |
| Multiagent DebateAgent Configuration=Multi-Agent2026.03 | 78.2 | 76.4 | 79.3 | |
| GPT-3.5RAG Protocol=No-RAG baselines2026.04 | 78.2 | — | — | |
| Qwen3-30BRAG Protocol=No-RAG baselines2026.04 | 77.7 | — | — | |
| RESTA-InstructModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 77.6 | — | — | |
| RESTA-InstructModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 77.4 | — | — | |
| GPT-4-baseRAG Protocol=No-RAG baselines, Few-shot settings=5-shot2026.04 | 77.4 | — | — | |
| Yi-34B (OpenMedLM)RAG Protocol=No-RAG baselines2026.04 | 77.3 | — | — | |
| MEDVRAG (full system)Memory Bank Usage=memory bank (full system)2026.04 | 77.2 | — | — | |
| MEDVRAG (+ iterative reasoning)Reasoning Rounds=iterative, Memory Bank Usage=no memory2026.04 | 77 | — | — | |
| MedAgentsAgent Configuration=Multi-Agent2026.03 | 76.8 | 75.1 | 80.3 | |
| RESTAModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 75.8 | — | — | |
| RESTAModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 75.7 | — | — | |
| Centralized Multi-Agent SystemAgent Configuration=Multi-Agent2026.03 | 75.3 | 74.1 | 76.6 | |
| MEDVRAG (single-round)Reasoning Rounds=single-round, Data format=page images2026.04 | 74.9 | — | — | |
| Qwen3-8BRAG Protocol=No-RAG baselines2026.04 | 74.7 | — | — | |
| OriginalModel=Llama-3.1-8B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 74.4 | — | — | |
| RESTAModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 74.2 | — | — | |
| Few-shot w/ CoT + SCAgent Configuration=Single-Agent, Prompting Setting=Few-shot, Reasoning Strategy=CoT + SC2026.03 | 73.8 | 72.1 | 74.1 | |
| MEDVRAG (no RAG)RAG Protocol=No RAG2026.04 | 73.7 | — | — | |
| OriginalModel=Qwen-2-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 73.6 | — | — | |
| MEDVRAG (text-only retrieval)Retrieval Type=text-only2026.04 | 73.3 | — | — | |
| OriginalModel=Qwen-2.5-7B-Instruct, Fine-tuning Context=PubMedQA2025.03 | 73.2 | — | — | |
| Fine-tunedModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 72.6 | — | — | |
| Few-shot w/ CoTAgent Configuration=Single-Agent, Prompting Setting=Few-shot, Reasoning Strategy=CoT2026.03 | 72.3 | 70.7 | 73.5 | |
| SafeMERGEModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 72.2 | — | — | |
| Zero-shot w/ CoT + SCAgent Configuration=Single-Agent, Prompting Setting=Zero-shot, Reasoning Strategy=CoT + SC2026.03 | 72.1 | 70.6 | 73 | |
| Few-shotAgent Configuration=Single-Agent, Prompting Setting=Few-shot2026.03 | 71.5 | 69.9 | 72.2 | |
| SafeLoRAModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 71.4 | — | — | |
| SafeInstructModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 71.2 | — | — | |
| Zero-shot w/ CoTAgent Configuration=Single-Agent, Prompting Setting=Zero-shot, Reasoning Strategy=CoT2026.03 | 70.7 | 69.2 | 72 | |
| MedRAG + GPT-4RAG Protocol=Text-only RAG2026.04 | 70.6 | — | — | |
| Zero-shotAgent Configuration=Single-Agent, Prompting Setting=Zero-shot2026.03 | 69.3 | 67.8 | 70.4 | |
| MedRAG + GPT-3.5RAG Protocol=Text-only RAG2026.04 | 67.4 | — | — | |
| RESTA-InstructModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 64.5 | — | — | |
| RESTAModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 57.1 | — | — | |
| OriginalModel=Llama-2-7B-Chat, Fine-tuning Context=PubMedQA2025.03 | 55.2 | — | — | |
| DataEnvGymBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=Synthesis2026.05 | 45.5 | — | — | |
| ConfidenceBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=AS2026.05 | 38.5 | — | — | |
| DiversityBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=AS2026.05 | 35.8 | — | — | |
| Gradient NormBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=AS2026.05 | 32.8 | — | — | |
| ProximityBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=AS2026.05 | 32.5 | — | — | |
| OriginalBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=Synthesis2026.05 | 31.3 | — | — | |
| RandomBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=Selection2026.05 | 30.8 | — | — | |
| Answer VarianceBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=AS2026.05 | 30.3 | — | — | |
| Qwen 3BBackbone=Qwen 3B, Method Category=Untrained model2026.05 | 28.5 | — | — | |
| Prismatic Syn.Backbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=Synthesis2026.05 | 28.2 | — | — | |
| FormatBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=AS2026.05 | 26.2 | — | — | |
| FilteredBackbone=Qwen 3B, Training Distribution=MedMCQA, Method Category=Selection2026.05 | 12.5 | — | — |