Medical Question Answering on MedMCQA
90.4AccuracyProtRLSearch
Evaluation Results
| Method | Links | |
|---|---|---|
| ProtRLSearchReinforcement Learning=true, Protein Pre-training=true2026.03 | 90.4 | |
| PADsource_model=Qwen3-32B, training_dataset=gsm8k2026.02 | 89.02 | |
| ProtRLSearchProtein Pre-training=false2026.03 | 88.5 | |
| PADsource_model=Gemma3-27B-it, training_dataset=gsm8k2026.02 | 88.48 | |
| PADsource_model=Qwen3-8B, training_dataset=gsm8k2026.02 | 88.36 | |
| DAsource_model=Qwen3-32B, training_dataset=gsm8k2026.02 | 87.49 | |
| RSFTsource_model=Qwen3-32B, training_dataset=gsm8k2026.02 | 87.04 | |
| RSFTsource_model=Qwen3-8B, training_dataset=gsm8k2026.02 | 86.88 | |
| DAsource_model=Qwen3-8B, training_dataset=gsm8k2026.02 | 86.35 | |
| RSFTsource_model=Gemma3-27B-it, training_dataset=gsm8k2026.02 | 86.35 | |
| DAsource_model=Gemma3-27B-it, training_dataset=gsm8k2026.02 | 86.28 | |
| ProtRLSearchReinforcement Learning=false2026.03 | 85.8 | |
| o3Size category=Large Models2025.07 | 83.3 | |
| Qwen3.5-397BBase Model=Qwen3.5-397B2026.05 | 83.2 | |
| o1Model Category=Proprietary2026.01 | 82.7 | |
| RandomModel=Llama 8B, Category=Selection2026.05 | 82.4 | |
| Kimi-K2.5Base Model=Kimi-K2.52026.05 | 82.3 | |
| AcquisitionSynthesis (Format)Model=Llama 8B, Category=AS2026.05 | 81.9 | |
| Qwen3.5-122BBase Model=Qwen3.5-122B2026.05 | 81.8 | |
| GLM-5.1Base Model=GLM-5.12026.05 | 81.3 | |
| Gemini 2.5 ProSize category=Large Models2025.07 | 81.1 | |
| AcquisitionSynthesis (Confidence)Model=Llama 8B, Category=AS2026.05 | 81 | |
| AcquisitionSynthesis (Gradient Norm)Model=Llama 8B, Category=AS2026.05 | 80.9 | |
| Deepseek-V3.2Base Model=Deepseek-V3.22026.05 | 80.7 | |
| GPT-4.12025.07 | 80.61 | |
| GPT-4.1(2025-04-14)Maximum output tokens=32,7682025.07 | 80.58 | |
| AcquisitionSynthesis (Answer Variance)Model=Llama 8B, Category=AS2026.05 | 80.3 | |
| Gemini 2.5 FlashSize category=Large Models2025.07 | 79.7 | |
| FilteredModel=Llama 8B, Category=Selection2026.05 | 79.3 | |
| Qwen3.5-35BBase Model=Qwen3.5-35B2026.05 | 79.2 | |
| DeepSeek R1Size category=Large Models2025.07 | 78.8 | |
| OriginalModel=Llama 8B, Category=Synthesis2026.05 | 78.8 | |
| AcquisitionSynthesis (Diversity)Model=Llama 8B, Category=AS2026.05 | 77.8 | |
| AcquisitionSynthesis (Proximity)Model=Llama 8B, Category=AS2026.05 | 77 | |
| HuatuoGPT-o1-72BParameter Scale=72B2025.04 | 76.76 | |
| SMCSMaximum output tokens=32,7682025.07 | 76.5 | |
| CascadeDebateModel Family=Llama-3.22026.04 | 76.33 | |
| DataEnvGymModel=Qwen 7B, Category=Synthesis2026.05 | 76.2 | |
| GPT-4o(2024-08-06)Maximum output tokens=32,7682025.07 | 76.17 | |
| GPT-4oSize category=Large Models2025.07 | 76.1 | |
| SEMA-RAGBackbone=kimi-k22026.05 | 76.07 | |
| Claude-3.5-Sonnet(2024-06-20)Maximum output tokens=32,7682025.07 | 76 | |
| HuatuoGPT-o1-72BMaximum output tokens=32,7682025.07 | 76 | |
| Untrained modelModel=Llama 8B2026.05 | 75.8 | |
| SMCS2025.07 | 75.69 | |
| FilteredModel=Qwen 7B, Category=Selection2026.05 | 75.4 | |
| OriginalModel=Qwen 7B, Category=Synthesis2026.05 | 75.3 | |
| AcquisitionSynthesis (Answer Variance)Model=Qwen 7B, Category=AS2026.05 | 75.3 | |
| HuatuoGPT-o1-70BParameter Scale=70B2025.04 | 75.23 | |
| AcquisitionSynthesis (Diversity)Model=Qwen 7B, Category=AS2026.05 | 75.2 | |
| SEMA-RAGBackbone=deepseek-v3.1, Retrieval Strategy=multi-round agentic retrieval2026.05 | 75.09 | |
| AcquisitionSynthesis (Confidence)Model=Qwen 7B, Category=AS2026.05 | 75 | |
| GPT-o3-mini(2025-01-31)Maximum output tokens=32,7682025.07 | 74.92 | |
| Self-MoA2025.07 | 74.92 | |
| GPT-OSS-120BBase Model=GPT-OSS-120B2026.05 | 74.9 | |
| MedAgentsBackbone=GPT-4, Evaluation Protocol=zero-shot setting2023.11 | 74.8 | |
| Claude-3.7-Sonnet(2025-02-19)Maximum output tokens=32,7682025.07 | 74.75 | |
| AcquisitionSynthesis (Proximity)Model=Qwen 7B, Category=AS2026.05 | 74.5 | |
| DRAGBackbone LLM=Phi-3.5-mini-instruct2025.06 | 74.4 | |
| OpenBioLLM-70BParameter Scale=70B2025.04 | 74.23 | |
| SEMA-RAGBackbone=qwen3-coder-plus2026.05 | 74.23 | |
| DRAGBackbone LLM=Llama-3.1-8B-Instruct2025.06 | 74.2 | |
| MedGemma 27BSize category=Small Models, test-time scaling=true2025.07 | 74.2 | |
| Qwen3.5-9B + MaRBase Model=Qwen3.5-9B, Optimization Method=MaR2026.05 | 74.2 | |
| i-MedRAGBackbone=kimi-k22026.05 | 74.13 | |
| AcquisitionSynthesis (Format)Model=Qwen 7B, Category=AS2026.05 | 74.1 | |
| OpenBioLLM 70BSize category=Large Models2025.07 | 74 | |
| MeerkatModel size=70B2024.03 | 73.9 | |
| MedCPTBackbone=kimi-k22026.05 | 73.85 | |
| Prismatic Syn.Model=Llama 8B, Category=Synthesis2026.05 | 73.8 | |
| GPT-4-baseRAG Protocol=No-RAG baselines, Few-shot settings=5-shot2026.04 | 73.7 | |
| DRAGBackbone LLM=Llama-3.2-3B-Instruct2025.06 | 73.6 | |
| GPT-4oModel Category=Proprietary2026.01 | 73.5 | |
| Qwen3.5-9B + DAPOBase Model=Qwen3.5-9B, Optimization Method=DAPO2026.05 | 73.5 | |
| Qwen3.5-9BBase Model=Qwen3.5-9B2026.05 | 73.4 | |
| MedRAGBackbone=kimi-k22026.05 | 73.2 | |
| DeepSeek-R1-Distill-Llama-70BMaximum output tokens=32,7682025.07 | 73.17 | |
| Few-shot CoT + SCBackbone=GPT-4, Evaluation Protocol=few-shot setting2023.11 | 73.1 | |
| CoTBackbone=qwen3-coder-plus2026.05 | 73.06 | |
| UltraMedical-70B-3Parameter Scale=70B2025.04 | 72.94 | |
| DRAGBackbone LLM=Qwen2.5-3B-Instruct2025.06 | 72.8 | |
| GPT-4Shots=5-shot2024.03 | 72.4 | |
| DRAGBackbone LLM=LLaMA-2-7B2025.06 | 72.4 | |
| DRAGBackbone LLM=Gemma-2-2B-it2025.06 | 72.4 | |
| Med-PaLM 2RAG Protocol=No-RAG baselines2026.04 | 72.3 | |
| Llama-3.3-70B-InstructMaximum output tokens=32,7682025.07 | 72.25 | |
| CoTBackbone=kimi-k22026.05 | 72.08 | |
| CoTBackbone=deepseek-v3.1, Retrieval Strategy=no retrieval2026.05 | 71.69 | |
| SEMA-RAGBackbone=gemini-2.0-flash2026.05 | 71.6 | |
| PulseMind-72BModel Category=Open-source, Model Scale=~72B2026.01 | 71.3 | |
| GLM-Z1-32B-0414Maximum output tokens=32,7682025.07 | 71.08 | |
| Qwen3-32BMaximum output tokens=32,7682025.07 | 70.92 | |
| Llama-3.3-Nemotron-Super-49B-v1Maximum output tokens=32,7682025.07 | 70.92 | |
| QwQ-32BMaximum output tokens=32,7682025.07 | 70.75 | |
| MA-PoPAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, Multi-Agent Strategy=Multi-Agent2026.05 | 70.67 | |
| AcquisitionSynthesis (Gradient Norm)Model=Qwen 7B, Category=AS2026.05 | 70.5 | |
| i-MedRAGBackbone=qwen3-coder-plus2026.05 | 70.26 | |
| PADsource_model=gemma3-27b-it, train_dataset=gsm8k, base_model=Qwen3-1.7B2026.02 | 70.2 | |
| Few-shotBackbone=GPT-4, Evaluation Protocol=few-shot setting2023.11 | 70.1 | |
| Zero-shot CoT + SCBackbone=GPT-4, Evaluation Protocol=zero-shot setting2023.11 | 70.1 |