List your company
ResearchFormerly WizWand

Medical Question Answering on MedQA US

90.42AccuracySEMA-RAG
33.927248.593663.2677.9264Apr 6, 2026Apr 12, 2026Apr 19, 2026Apr 26, 2026May 2, 2026May 9, 2026May 16, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SEMA-RAGBackbone=gemini-2.0-flash, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.0590.42
SEMA-RAGBackbone=deepseek-v3.1, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.0589.95
SEMA-RAGBackbone=kimi-k2, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.0586.41
SEMA-RAGBackbone=qwen3-coder-plus, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.0586.17
MedRAGBackbone=gemini-2.0-flash, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.0583.19
i-MedRAGBackbone=kimi-k2, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.0581.78
MedRAGBackbone=kimi-k2, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.0581.54
MedRAGBackbone=qwen3-coder-plus, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.0581.54
MedCPTBackbone=kimi-k2, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.0580.68
CoTBackbone=kimi-k2, Retrieval Strategy=no retrieval, Setting=zero-shot2026.0577.85
i-MedRAGBackbone=gemini-2.0-flash, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.0577.69
CoTBackbone=deepseek-v3.1, Retrieval Strategy=no retrieval, Setting=zero-shot2026.0577.53
i-MedRAGBackbone=qwen3-coder-plus, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.0577.38
MedRAGBackbone=deepseek-v3.1, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.0577.14
CoTBackbone=qwen3-coder-plus, Retrieval Strategy=no retrieval, Setting=zero-shot2026.0576.9
MedCPTBackbone=qwen3-coder-plus, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.0575.26
i-MedRAGBackbone=deepseek-v3.1, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.0574.78
MedCPTBackbone=deepseek-v3.1, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.0573.84
MedCPTBackbone=gemini-2.0-flash, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.0570.54
CoTBackbone=gemini-2.0-flash, Retrieval Strategy=no retrieval, Setting=zero-shot2026.0565.12
SEMA-RAGBackbone=glm-4.0-flash, Retrieval Strategy=multi-round agentic retrieval, Setting=zero-shot2026.0563.79
MedRAGBackbone=glm-4.0-flash, Retrieval Strategy=retrieval-fusion, Setting=zero-shot2026.0560.88
MedCPTBackbone=glm-4.0-flash, Retrieval Strategy=single-round retrieval, Setting=zero-shot2026.0558.21
i-MedRAGBackbone=glm-4.0-flash, Retrieval Strategy=iterative retrieval, Setting=zero-shot2026.0553.57
CHRGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0453.1
ThinkQEGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0452.2
HyDE (H+ only)Generator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0451.6
CHRGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0451.5
CoTBackbone=glm-4.0-flash, Retrieval Strategy=no retrieval, Setting=zero-shot2026.0550.43
CSQEGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0449.6
CHRGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0449.2
Query2docGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0446.9
CSQEGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0446.5
Query2docGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0446.2
HyDE (H+ only)Generator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0446
Standard RAGGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0446
ThinkQEGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0445.6
HyDE (H+ only)Generator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0444.6
ThinkQEGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0443
CSQEGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0442.2
Query2docGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0441.5
Standard RAGGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0439.7
Standard RAGGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.0436.1