Question Answering on HotpotQA (test) (EM and F1 Metrics)
56.2EMVEROIC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VEROICInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 56.2 | 72.8 | |
| RISK PREDICTORInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 55.9 | 72.3 | |
| VEROICInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 55.8 | 72.1 | |
| CONFIDENCE-THRESHOLDInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 55.6 | 71.9 | |
| OUTPUT-TRIGGERInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 55.3 | 71.6 | |
| RISK PREDICTORInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 55.3 | 71.9 | |
| CONFIDENCE-THRESHOLDInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 54.9 | 71.6 | |
| PERIODICInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 54.7 | 70.8 | |
| OUTPUT-TRIGGERInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 54.6 | 71.3 | |
| RANDOMInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 53.8 | 70.6 | |
| PERIODICInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 53.3 | 70.3 | |
| RANDOMInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 53.1 | 70.2 | |
| FIXEDInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 51.6 | 68.7 | |
| FIXEDInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 51.2 | 68.3 | |
| R3AGPipeline Type=Route2026.04 | 29.13 | 40.73 | |
| IRCoTPipeline Type=Loop2026.04 | 29.05 | 40.52 | |
| Adaptive-RAGPipeline Type=Conditional2026.04 | 28.87 | 40.54 | |
| BGE-largePipeline Type=Sequential2026.04 | 27.54 | 38.6 | |
| Oracle Single BestPipeline Type=Conditional2026.04 | 27.54 | 38.6 | |
| E5-largePipeline Type=Sequential2026.04 | 26.87 | 37.87 | |
| Qwen3-embedding-4bPipeline Type=Sequential2026.04 | 26.64 | 37.62 | |
| E5-base-v2Pipeline Type=Sequential2026.04 | 25.31 | 35.75 | |
| BGE-m3Pipeline Type=Sequential2026.04 | 24.73 | 35.01 | |
| Qwen3-embedding-0.6bPipeline Type=Sequential2026.04 | 24.42 | 34.63 | |
| RouterRetrieverPipeline Type=Route2026.04 | 24.37 | 34.98 | |
| AAR-contriever-kiltPipeline Type=Sequential2026.04 | 23.04 | 33.37 | |
| DIVER-Retriever-0.6bPipeline Type=Sequential2026.04 | 22.75 | 32.78 | |
| RandomPipeline Type=Conditional2026.04 | 22.13 | 32.09 | |
| FLAREPipeline Type=Loop2026.04 | 20.77 | 28.03 | |
| LTRRPipeline Type=Route2026.04 | 19.86 | 28.64 | |
| NaivePipeline Type=–2026.04 | 19.16 | 26.96 | |
| BM25Pipeline Type=Sequential2026.04 | 17.87 | 26.15 |