Medical Question Answering on MedQA US
90.42AccuracySEMA-RAG
Evaluation Results
| Method | Links | |
|---|---|---|
| SEMA-RAGBackbone=gemini-2.0-flash, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.05 | 90.42 | |
| SEMA-RAGBackbone=deepseek-v3.1, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.05 | 89.95 | |
| SEMA-RAGBackbone=kimi-k2, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.05 | 86.41 | |
| SEMA-RAGBackbone=qwen3-coder-plus, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.05 | 86.17 | |
| MedRAGBackbone=gemini-2.0-flash, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.05 | 83.19 | |
| i-MedRAGBackbone=kimi-k2, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.05 | 81.78 | |
| MedRAGBackbone=kimi-k2, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.05 | 81.54 | |
| MedRAGBackbone=qwen3-coder-plus, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.05 | 81.54 | |
| MedCPTBackbone=kimi-k2, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.05 | 80.68 | |
| CoTBackbone=kimi-k2, Retrieval Strategy=no retrieval, Setting=zero-shot2026.05 | 77.85 | |
| i-MedRAGBackbone=gemini-2.0-flash, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.05 | 77.69 | |
| CoTBackbone=deepseek-v3.1, Retrieval Strategy=no retrieval, Setting=zero-shot2026.05 | 77.53 | |
| i-MedRAGBackbone=qwen3-coder-plus, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.05 | 77.38 | |
| MedRAGBackbone=deepseek-v3.1, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.05 | 77.14 | |
| CoTBackbone=qwen3-coder-plus, Retrieval Strategy=no retrieval, Setting=zero-shot2026.05 | 76.9 | |
| MedCPTBackbone=qwen3-coder-plus, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.05 | 75.26 | |
| i-MedRAGBackbone=deepseek-v3.1, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.05 | 74.78 | |
| MedCPTBackbone=deepseek-v3.1, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.05 | 73.84 | |
| MedCPTBackbone=gemini-2.0-flash, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.05 | 70.54 | |
| CoTBackbone=gemini-2.0-flash, Retrieval Strategy=no retrieval, Setting=zero-shot2026.05 | 65.12 | |
| SEMA-RAGBackbone=glm-4.0-flash, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.05 | 63.79 | |
| MedRAGBackbone=glm-4.0-flash, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.05 | 60.88 | |
| MedCPTBackbone=glm-4.0-flash, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.05 | 58.21 | |
| i-MedRAGBackbone=glm-4.0-flash, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.05 | 53.57 | |
| CHRGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 53.1 | |
| ThinkQEGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 52.2 | |
| HyDE (H+ only)Generator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 51.6 | |
| CHRGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 51.5 | |
| CoTBackbone=glm-4.0-flash, Retrieval Strategy=no retrieval, Setting=zero-shot2026.05 | 50.43 | |
| CSQEGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 49.6 | |
| CHRGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 49.2 | |
| Query2docGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 46.9 | |
| CSQEGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 46.5 | |
| Query2docGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 46.2 | |
| HyDE (H+ only)Generator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 46 | |
| Standard RAGGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 46 | |
| ThinkQEGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 45.6 | |
| HyDE (H+ only)Generator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 44.6 | |
| ThinkQEGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 43 | |
| CSQEGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 42.2 | |
| Query2docGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 41.5 | |
| Standard RAGGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 39.7 | |
| Standard RAGGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 36.1 |