Question Answering on NaturalQA
38.92EMDTR
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DTRGenerator=Qwen2.5-72B-Instruct, Threshold u=0.0012026.01 | 38.92 | 51.28 | 88.1 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=3, Uncertainty threshold (u)=0.0012026.01 | 38.84 | 49.22 | 94.8 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=5, Uncertainty threshold (u)=0.0012026.01 | 38.5 | 48.63 | 94.8 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=3, Uncertainty threshold (u)=0.0052026.01 | 38.12 | 48.64 | 90.4 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Threshold u=0.0012026.01 | 38.03 | 47.83 | 94.8 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=5, Uncertainty threshold (u)=0.0052026.01 | 37.95 | 48.13 | 90.4 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=3, Uncertainty threshold (u)=0.012026.01 | 37.92 | 48.45 | 87.5 | |
| DTRGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=5, Uncertainty threshold (u)=0.012026.01 | 37.67 | 47.95 | 87.5 | |
| Standard RAGGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=32026.01 | 37.56 | 47.29 | — | |
| CoTGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=52026.01 | 37.17 | 47.46 | — | |
| Q2DGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=52026.01 | 36.87 | 47.15 | — | |
| HyDEGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=52026.01 | 36.7 | 46.35 | — | |
| Standard RAGGenerator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=52026.01 | 36.45 | 46.23 | — | |
| CoTGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=32026.01 | 36.18 | 45.89 | — | |
| Q2DGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=32026.01 | 35.93 | 45.87 | — | |
| HyDEGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=32026.01 | 35.32 | 44.92 | — | |
| Standard RAGGenerator=Qwen2.5-7B-Instruct, Retriever=bge-large-en-v1.5, top-k=32026.01 | 34.29 | 44 | — | |
| LLM JudgeGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=3, Judge Model=72B2026.01 | 28.53 | 37.66 | 43.7 | |
| LLM Judge (72B)Generator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=52026.01 | 27.89 | 36.78 | 43.7 | |
| LLM Judge (72B)Generator=Qwen2.5-7B-Instruct, Retriever=bge-large-en-v1.52026.01 | 26.79 | 35.64 | 43.7 | |
| No RetrievalGenerator=Qwen2.5-7B-Instruct, Retriever=None, Top-k retrievals=02026.01 | 17.4 | 24.95 | — | |
| LLM JudgeGenerator=Qwen2.5-7B-Instruct, Retriever=e5, Top-k retrievals=3, Judge Model=7B2026.01 | 17.4 | 24.95 | 0 | |
| No RetrievalGenerator=Qwen2.5-7B-Instruct, Retriever=None2026.01 | 17.4 | 24.95 | — | |
| LLM Judge (7B)Generator=Qwen2.5-7B-Instruct, Retriever=bge-large-en-v1.52026.01 | 17.4 | 24.95 | 0 | |
| No RetrievalGenerator=Qwen2.5-7B-Instruct, Retriever=None2026.01 | 17.4 | 24.95 | — | |
| LLM Judge (7B)Generator=Qwen2.5-7B-Instruct, Retriever=bge, Retrieval depth (top-k)=52026.01 | 17.4 | 24.95 | 0 |