Multi-hop Reasoning on HotpotQA IRCoT (500 samples) (test)
54.6ACCActiShade
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ActiShadeBackbone=Llama-3-8B-Instruct, Retrieval Mode=Active/Iterative retrieval2026.01 | 54.6 | 56.33 | |
| ActiShadeBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Active/Iterative retrieval2026.01 | 53.97 | 57.45 | |
| DRAGINBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 51.21 | 54.3 | |
| DRAGINBackbone=Llama-3-8B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 50.87 | 52.52 | |
| Iter-RetGenBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 49.03 | 51.04 | |
| FLAREBackbone=Llama-3-8B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 48.45 | 50.4 | |
| Iter-RetGenBackbone=Llama-3-8B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 48.23 | 49.41 | |
| ActiShadeBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Active/Iterative retrieval2026.01 | 48.2 | 55.45 | |
| IRCoTBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 47.98 | 50.5 | |
| SelfASKBackbone=Llama-3-8B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 47.1 | 48.7 | |
| Direct-RBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Single-round retrieval2026.01 | 46.2 | 48.11 | |
| DRAGINBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 46.1 | 50.3 | |
| SelfASKBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 45.97 | 47.19 | |
| Iter-RetGenBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 44.4 | 48.24 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 43.79 | 48.13 | |
| Direct-RBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Single-round retrieval2026.01 | 43 | 47.99 | |
| SelfASKBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 42.18 | 46.1 | |
| FLAREBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 42.14 | 44.87 | |
| FLAREBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 40.19 | 42.03 | |
| IRCoTBackbone=Llama-3-8B-Instruct, Retrieval Mode=Multi-round retrieval2026.01 | 40.1 | 47.03 | |
| Direct-RBackbone=Llama-3-8B-Instruct, Retrieval Mode=Single-round retrieval2026.01 | 37.7 | 44.89 | |
| CoTBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=None2026.01 | 29.4 | 34.92 | |
| CoTBackbone=Llama-3-8B-Instruct, Retrieval Mode=None2026.01 | 29 | 34.09 | |
| DirectBackbone=Qwen2.5-14B-Instruct, Retrieval Mode=None2026.01 | 27.2 | 32.94 | |
| DirectBackbone=Llama-3-8B-Instruct, Retrieval Mode=None2026.01 | 22.4 | 25.34 | |
| CoTBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=None2026.01 | 22 | 27.61 | |
| DirectBackbone=Qwen2.5-7B-Instruct, Retrieval Mode=None2026.01 | 19.4 | 19.52 |