Question Answering on MuSiQue (EM, F1)
39.7EMIRCoT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| IRCoTInference LLM=Gemini-2.5-flash2025.08 | 39.7 | 49.8 | |
| T^2RAGInference LLM=Gemini-2.5-flash2025.08 | 39.1 | 49.1 | |
| Ψ-RAG ⍋Type=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R), Abstract Type=summative2026.05 | 38.7 | 48.9 | |
| Ψ-RAG øType=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R), Abstract Type=keyword2026.05 | 37.9 | 47.83 | |
| HippoRAG 2Type=Graph-RAG2026.05 | 37.5 | 48 | |
| T^2RAGInference LLM=GPT-4o-mini2025.08 | 34.3 | 45.6 | |
| HippoRAG2Inference LLM=GPT-4o-mini2025.08 | 34.1 | 48.1 | |
| IRCoTInference LLM=GPT-4o-mini2025.08 | 34.1 | 47.6 | |
| HippoRAG 2 + Q3RType=Graph-RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 32.9 | 43.27 | |
| RAPTORInference LLM=Gemini-2.5-flash2025.08 | 31.8 | 39.7 | |
| StandardInference LLM=GPT-4o-mini2025.08 | 31.2 | 44.3 | |
| VECTORRAGLLM=GPT-OSS-120B2026.05 | 30.4 | 43.6 | |
| DPR + IRCoT + Q3RType=Traditional RAG, Reranker=Qwen3-Reranker-8B (Q3R), Iterative Retrieval Protocol=IRCoT2026.05 | 29.7 | 37.36 | |
| HippoRAG2Inference LLM=Gemini-2.5-flash2025.08 | 29.1 | 38.3 | |
| RAPTORInference LLM=GPT-4o-mini2025.08 | 28.6 | 40.8 | |
| BM25 + DPR + Q3RType=Traditional RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 28.3 | 34.81 | |
| StandardInference LLM=Gemini-2.5-flash2025.08 | 28.1 | 36.2 | |
| GraphRAGType=Graph-RAG2026.05 | 25.8 | 30.7 | |
| M2RModel Family=Mistral-7B-Instruct2026.04 | 25.5 | — | |
| WIKONTICLLM=Qwen3-30B-A3B2026.05 | 25 | 37.4 | |
| OAKLLM=Qwen3-30B-A3B2026.05 | 24.4 | 36.5 | |
| ReSearchModel Family=Mistral-7B-Instruct2026.04 | 23.7 | — | |
| OAK+MENDLLM=Qwen3-30B-A3B2026.05 | 23.3 | 35.2 | |
| OAKLLM=GPT-OSS-120B2026.05 | 22.6 | 31.1 | |
| M2RModel Family=Llama-3.1-8B-Instruct2026.04 | 22.1 | — | |
| HIPPORAGV2LLM=Qwen3-30B-A3B2026.05 | 22 | 33.9 | |
| IN-CONTEXTLLM=Qwen3-30B-A3B2026.05 | 22 | 34 | |
| KGGENLLM=GPT-OSS-120B2026.05 | 21.9 | 28.6 | |
| OAK+MENDLLM=GPT-OSS-120B2026.05 | 21.5 | 29.9 | |
| SUREModel Family=Mistral-7B-Instruct2026.04 | 21.2 | — | |
| ReSearchModel Family=Llama-3.1-8B-Instruct2026.04 | 20.9 | — | |
| WIKONTICLLM=GPT-OSS-120B2026.05 | 19.9 | 28 | |
| HIPPORAGV2LLM=GPT-OSS-120B2026.05 | 19.4 | 36.2 | |
| KGGENLLM=Qwen3-30B-A3B2026.05 | 19.2 | 29.2 | |
| DPRType=Traditional RAG2026.05 | 18.7 | 24.03 | |
| SUREModel Family=Llama-3.1-8B-Instruct2026.04 | 18.7 | — | |
| COFTModel Family=Mistral-7B-Instruct2026.04 | 18.5 | — | |
| IN-CONTEXTLLM=GPT-OSS-120B2026.05 | 17.5 | 23.7 | |
| RAPTOR + Q3RType=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 16.6 | 21.1 | |
| NORInference LLM=Gemini-2.5-flash2025.08 | 16.3 | 26.5 | |
| COFTModel Family=Llama-3.1-8B-Instruct2026.04 | 15.8 | — | |
| BM25Inference LLM=GPT-4o-mini2025.08 | 15.3 | 21.1 | |
| VECTORRAGLLM=Qwen3-30B-A3B2026.05 | 14 | 23.2 | |
| BM25Type=Traditional RAG2026.05 | 13 | 17.96 | |
| ZEROSHOTLLM=GPT-OSS-120B2026.05 | 12.9 | 24.1 | |
| Naive RAGModel Family=Mistral-7B-Instruct2026.04 | 11.3 | — | |
| NORInference LLM=GPT-4o-mini2025.08 | 10.2 | 20.3 | |
| Naive RAGModel Family=Llama-3.1-8B-Instruct2026.04 | 9.8 | — | |
| ParaSet+SetCEAugmentation=∪ CE2026.07 | 9.3 | 15.7 | |
| LLM2026.07 | 9.1 | 14.3 | |
| SetCE2026.07 | 9 | 16.2 | |
| CE2026.07 | 8.5 | 15.9 | |
| BM25Inference LLM=Gemini-2.5-flash2025.08 | 7.9 | 10.7 | |
| Naive GenerationModel Family=Mistral-7B-Instruct2026.04 | 7.7 | — | |
| ParaSet+SetCE2026.07 | 7.3 | 14.1 | |
| Naive GenerationModel Family=Llama-3.1-8B-Instruct2026.04 | 4.4 | — | |
| ZEROSHOTLLM=Qwen3-30B-A3B2026.05 | 2.3 | 9.4 |