Medical Question Answering on BioASQ
88.67AccuracySEMA-RAG
Evaluation Results
| Method | Links | |
|---|---|---|
| SEMA-RAGBackbone=kimi-k22026.05 | 88.67 | |
| SEMA-RAGBackbone=gemini-2.0-flash2026.05 | 88.19 | |
| CoTBackbone=kimi-k22026.05 | 85.76 | |
| MedRAGBackbone=kimi-k22026.05 | 85.6 | |
| i-MedRAGBackbone=kimi-k22026.05 | 83.17 | |
| SEMA-RAGBackbone=qwen3-coder-plus2026.05 | 83.01 | |
| SEMA-RAGBackbone=deepseek-v3.12026.05 | 82.85 | |
| i-MedRAGBackbone=qwen3-coder-plus2026.05 | 82.52 | |
| CoTBackbone=qwen3-coder-plus2026.05 | 81.72 | |
| MedCPTBackbone=kimi-k22026.05 | 81.39 | |
| ReFilterBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.02 | 80.74 | |
| CHRGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 80.7 | |
| i-MedRAGBackbone=deepseek-v3.12026.05 | 80.58 | |
| CoTBackbone=deepseek-v3.12026.05 | 80.1 | |
| S-RAGBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.02 | 79.29 | |
| DyPRAGBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.02 | 79.29 | |
| PRAGBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.02 | 79.13 | |
| ReFilterBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.02 | 78.8 | |
| MedRAGBackbone=deepseek-v3.12026.05 | 78.48 | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Zero-shot evaluation=true2026.02 | 78.16 | |
| i-MedRAGBackbone=gemini-2.0-flash2026.05 | 77.99 | |
| S-RAGBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.02 | 77.18 | |
| VanillaBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.02 | 76.54 | |
| MedCPTBackbone=deepseek-v3.12026.05 | 76.38 | |
| PRAGBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.02 | 76.05 | |
| DyPRAGBackbone=LLaMA-3-8B-Instruct, Zero-shot evaluation=true2026.02 | 75.89 | |
| MedCPTBackbone=qwen3-coder-plus2026.05 | 75.89 | |
| CHRGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 75.6 | |
| Standard RAGGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 74.3 | |
| SEMA-RAGBackbone=glm-4.0-flash2026.05 | 72.98 | |
| MedRAGBackbone=gemini-2.0-flash2026.05 | 72.65 | |
| MedRAGBackbone=qwen3-coder-plus2026.05 | 72.33 | |
| CSQEGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 72.2 | |
| ReFilterBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.02 | 71.04 | |
| MedCPTBackbone=gemini-2.0-flash2026.05 | 70.06 | |
| PRAGBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.02 | 69.9 | |
| CoTBackbone=gemini-2.0-flash2026.05 | 68.45 | |
| DyPRAGBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.02 | 67.96 | |
| VanillaBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.02 | 66.18 | |
| S-RAGBackbone=Qwen2.5-1.5B-Instruct, Zero-shot evaluation=true2026.02 | 66.18 | |
| Query2docGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 65.2 | |
| i-MedRAGBackbone=glm-4.0-flash2026.05 | 64.72 | |
| CHRGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 63.1 | |
| MedRAGBackbone=glm-4.0-flash2026.05 | 62.78 | |
| MedCPTBackbone=glm-4.0-flash2026.05 | 60.84 | |
| Standard RAGGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 60.5 | |
| CoTBackbone=glm-4.0-flash2026.05 | 58.9 | |
| Query2docGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 58.6 | |
| CSQEGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 56.5 | |
| ThinkQEGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 52.1 | |
| HyDE (H+ only)Generator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 51.3 | |
| HyDE (H+ only)Generator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 48.4 | |
| Query2docGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 46.9 | |
| CSQEGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 44.8 | |
| ThinkQEGenerator=Gemma-2-9B-It, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 44.5 | |
| ThinkQEGenerator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 43.4 | |
| HyDE (H+ only)Generator=Qwen2.5-7B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 42.2 | |
| Standard RAGGenerator=Llama-3-8B-Instruct, Hypothesis Generator=Qwen2.5-72B-Instruct, Retriever=MedCPT, Corpus=MedCorp2026.04 | 41.8 | |
| ReFilterBackbone=LLaMA-3.2-1B-Instruct, Zero-shot evaluation=true2026.02 | 37.22 | |
| VanillaBackbone=LLaMA-3.2-1B-Instruct, Zero-shot evaluation=true2026.02 | 36.57 | |
| PRAGBackbone=LLaMA-3.2-1B-Instruct, Zero-shot evaluation=true2026.02 | 35.44 | |
| DyPRAGBackbone=LLaMA-3.2-1B-Instruct, Zero-shot evaluation=true2026.02 | 35.11 | |
| S-RAGBackbone=LLaMA-3.2-1B-Instruct, Zero-shot evaluation=true2026.02 | 34.47 |