Medical Question Answering on MedBullets
84.2AccuracyMulti-Agent Medical Decision Consensus Matrix System
Evaluation results
| Method | Links | |
|---|---|---|
| Multi-Agent Medical Decision Consensus Matrix System2025.12 | 84.2 | |
| BFRSModel=o3 Mini2025.11 | 82.5 | |
| Zero-Shot PredictModel=o3 Mini2025.11 | 81.5 | |
| Zero-Shot CoTModel=o3 Mini2025.11 | 81.5 | |
| MIPROv2Model=o3 Mini2025.11 | 81.5 | |
| HELM BaselineModel=o3 Mini2025.11 | 81.2 | |
| TeamMedAgents2025.12 | 80.3 | |
| Claude Sonnet 4.5Backbone=Claude Sonnet 4.52026.07 | 80 | |
| MDAgents2025.12 | 79.6 | |
| Weighted VotingMulti-Agent Aggregation Method=Weighted Voting2025.12 | 78.3 | |
| Majority VotingMulti-Agent Aggregation Method=Majority Voting2025.12 | 77.1 | |
| Borda CountMulti-Agent Aggregation Method=Borda Count2025.12 | 76.8 | |
| Chain-of-ThoughtPrompting Strategy=Chain-of-Thought2025.12 | 75.9 | |
| MedGemma 27BBackbone=MedGemma 27B2026.07 | 75 | |
| Single-AgentEvaluation Protocol=Few-shot2025.12 | 74.2 | |
| MIPROv2Model=GPT 4o2025.11 | 73.4 | |
| DeepSeek-R1Backbone=DeepSeek-R12026.07 | 73.2 | |
| BFRSModel=GPT 4o2025.11 | 72.7 | |
| MIPROv2Model=Claude 3.7 Sonnet2025.11 | 72.7 | |
| Single-AgentEvaluation Protocol=Zero-shot2025.12 | 72.5 | |
| Zero-Shot CoTModel=GPT 4o2025.11 | 72.1 | |
| Zero-Shot CoTModel=Claude 3.7 Sonnet2025.11 | 71.8 | |
| Zero-Shot PredictModel=GPT 4o2025.11 | 71.4 | |
| BFRSModel=Claude 3.7 Sonnet2025.11 | 71.4 | |
| HELM BaselineModel=GPT 4o2025.11 | 71.1 | |
| Zero-Shot CoTModel=Gemini 2.0 Flash2025.11 | 70.8 | |
| MIPROv2Model=Gemini 2.0 Flash2025.11 | 69.8 | |
| Full FTBase model=gpt-oss 20B, Trainable Parameters=100.0%2026.04 | 69.7 | |
| BFRSModel=Gemini 2.0 Flash2025.11 | 69.5 | |
| MPTBase model=gpt-oss 20B, Trainable Parameters=<0.05%2026.04 | 68.9 | |
| BFRSModel=Llama 3.3 70B2025.11 | 68.5 | |
| LoRABase model=gpt-oss 20B, Trainable Parameters=~2.50%2026.04 | 67.3 | |
| Zero-Shot CoTModel=Llama 3.3 70B2025.11 | 65.9 | |
| Zero-Shot PredictModel=Claude 3.7 Sonnet2025.11 | 65.3 | |
| MIPROv2Model=Llama 3.3 70B2025.11 | 65.3 | |
| Full FTBase model=Meditron3 8B, Trainable Parameters=100.0%2026.04 | 65.1 | |
| HELM BaselineModel=Claude 3.7 Sonnet2025.11 | 64.9 | |
| Zero-Shot PredictModel=Gemini 2.0 Flash2025.11 | 64.9 | |
| MPTBase model=Meditron3 8B, Trainable Parameters=<0.05%2026.04 | 64.3 | |
| Zero-Shot PredictModel=Llama 3.3 70B2025.11 | 63.6 | |
| PTBase model=gpt-oss 20B, Trainable Parameters=<0.05%2026.04 | 63.4 | |
| HELM BaselineModel=Gemini 2.0 Flash2025.11 | 63 | |
| Full FTBase model=LLaMA 3.1 8B, Trainable Parameters=100.0%2026.04 | 62.7 | |
| LoRABase model=Meditron3 8B, Trainable Parameters=~2.50%2026.04 | 62.4 | |
| Qwen3 8BBackbone=Qwen3 8B2026.07 | 61.8 | |
| MPTBase model=LLaMA 3.1 8B, Trainable Parameters=<0.05%2026.04 | 61.6 | |
| HELM BaselineModel=Llama 3.3 70B2025.11 | 60.7 | |
| LoRABase model=LLaMA 3.1 8B, Trainable Parameters=~2.50%2026.04 | 59.9 | |
| MA-RAG-extBackbone=Qwen3-8B, Ranking Agent=extrinsic verification2026.02 | 59.1 | |
| PTBase model=Meditron3 8B, Trainable Parameters=<0.05%2026.04 | 58.3 | |
| MA-RAG-intBackbone=Qwen3-8B, Ranking Agent=intrinsic uncertainty2026.02 | 57.1 | |
| Multi-RefineBackbone=Qwen3-8B2026.02 | 55.8 | |
| PTBase model=LLaMA 3.1 8B, Trainable Parameters=<0.05%2026.04 | 55.6 | |
| FaithMed (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 55.6 | |
| UltraMedical-3.1-8BBackbone=UltraMedical-3.1-8B2026.02 | 54.5 | |
| Zero-Shot CoTModel=Qwen3 4B2025.11 | 52.6 | |
| FL-RAGBackbone=Qwen3-8B2026.02 | 52.3 | |
| Llama 3.1 8B InstructBackbone=Llama 3.1 8B2026.07 | 52.2 | |
| MedRAG (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 52.1 | |
| SCBackbone=Qwen3-8B2026.02 | 51.9 | |
| FLAREBackbone=Qwen3-8B2026.02 | 51.9 | |
| BFRSModel=Qwen3 4B2025.11 | 51.9 | |
| FS-RAGBackbone=Qwen3-8B2026.02 | 51.3 | |
| Qwen3-8BBackbone=Qwen3-8B2026.02 | 51 | |
| Gemma 3 12BBackbone=Gemma 3 12B2026.07 | 51 | |
| HuatuoGPT-o1-8BBackbone=HuatuoGPT-o1-8B2026.02 | 50.6 | |
| CoTBackbone=Qwen3-8B2026.02 | 50.6 | |
| MIPROv2Model=Qwen3 4B2025.11 | 50.3 | |
| Agentic Search + GRPO (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 49.8 | |
| SR-RAGBackbone=Qwen3-8B2026.02 | 49.4 | |
| TC-RAGBackbone=Qwen3-8B2026.02 | 49 | |
| Agentic Search (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 47.3 | |
| Llama-3.1-8BBackbone=Llama-3.1-8B2026.02 | 47.1 | |
| Base (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 46.3 | |
| Zero-Shot PredictModel=Qwen3 4B2025.11 | 43.5 | |
| HELM BaselineModel=Qwen3 4B2025.11 | 41.9 | |
| MedRAG (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 40.8 | |
| FaithMed (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 38.5 | |
| Agentic Search (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 36 | |
| Base (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 35.7 | |
| Agentic Search + GRPO (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 31 |