Multi-hop Question Answering on 2WikiMultiHopQA (dev)
88.658Exact Match AccuracyDense-aware HKVM controller
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Dense-aware HKVM controllervariant_type=Dense-aware controller, training_signals=out-of-fold HKVM2026.06 | 88.658 | 88.846 | 100 | |
| Controller HKVM-onlyvariant_type=Source-only controller control, training_signals=out-of-fold HKVM2026.06 | 88.205 | 88.415 | 100 | |
| Learned HKVM calibrationvariant_type=Reference predictor, training_signals=out-of-fold HKVM2026.06 | 88.181 | 88.391 | 99.03 | |
| ColBERTv2variant_type=Reference predictor2026.06 | 77.195 | 77.763 | 100 | |
| Controller dense-onlyvariant_type=Source-only controller control, training_signals=out-of-fold HKVM2026.06 | 77.107 | 77.679 | 100 | |
| MCR+SC@3Retrieval=yes, LLM=code-davinci-002, Reasoning Chains=152023.04 | 68.6 | — | — | |
| MCRRetrieval=yes, LLM=code-davinci-002, Reasoning Chains=52023.04 | 67.9 | — | — | |
| PASSProcess signal=G-OPD, Standardization operator=Masked-Norm, Actor model=Qwen2.5-7B-Instruct, Teacher model=Qwen2.5-32B-Instruct2026.06 | 67.19 | — | — | |
| PASSProcess signal=OPD, Standardization operator=Masked-Norm, Actor model=Qwen2.5-7B-Instruct, Teacher model=Qwen2.5-32B-Instruct2026.06 | 66.81 | — | — | |
| IR-COTRetrieval=yes, LLM=code-davinci-002, Reasoning Chains=12023.04 | 65.2 | — | — | |
| ARCOBackbone=Qwen3-4B2026.06 | 65.2 | 69.38 | — | |
| Self-Ask (ours)Retrieval=yes, LLM=code-davinci-002, Reasoning Chains=12023.04 | 63.8 | — | — | |
| GRPOProcess signal=G-OPD, Standardization operator=Masked-Norm, Actor model=Qwen2.5-7B-Instruct, Teacher model=Qwen2.5-32B-Instruct2026.06 | 63.69 | — | — | |
| Search-R1Backbone=Qwen3-4B2026.06 | 63.4 | 67.95 | — | |
| GRPOProcess signal=OPD, Standardization operator=Masked-Norm, Actor model=Qwen2.5-7B-Instruct, Teacher model=Qwen2.5-32B-Instruct2026.06 | 63.25 | — | — | |
| R1-SearcherBackbone=Qwen3-4B2026.06 | 62 | 67.19 | — | |
| AgentPRMBackbone=Qwen3-4B2026.06 | 62 | 65.32 | — | |
| RaRBackbone=Qwen3-4B2026.06 | 61.4 | 67.06 | — | |
| RLERBackbone=Qwen3-4B2026.06 | 61 | 66.06 | — | |
| ARCOBackbone=Llama-3.2-3B2026.06 | 59.6 | 63.67 | — | |
| CARMOBackbone=Qwen3-4B2026.06 | 59.4 | 65.77 | — | |
| Search-R1Backbone=Llama-3.2-3B2026.06 | 58.2 | 62.17 | — | |
| R1-SearcherBackbone=Llama-3.2-3B2026.06 | 57.8 | 61.48 | — | |
| RaRBackbone=Llama-3.2-3B2026.06 | 53.8 | 57.88 | — | |
| Self-AskRetrieval=yes, LLM=text-davinci-002, Reasoning Chains=12023.04 | 52.6 | — | — | |
| CARMOBackbone=Llama-3.2-3B2026.06 | 51.6 | 58.15 | — | |
| RLERBackbone=Llama-3.2-3B2026.06 | 50 | 55 | — | |
| AgentPRMBackbone=Llama-3.2-3B2026.06 | 49.6 | 54.63 | — | |
| Base ModelBackbone=Qwen3-4B, Prompting=zero-shot2026.06 | 47.8 | 53.95 | — | |
| Self-ask + SearchModel=Davinci-0022022.10 | 40.1 | — | — | |
| Self-askModel=Davinci-0022022.10 | 30 | — | — | |
| Chain of ThoughtModel=Davinci-0022022.10 | 29.8 | — | — | |
| Search + postproc.Model=Davinci-0022022.10 | 26.3 | — | — | |
| Base ModelBackbone=Llama-3.2-3B, Prompting=zero-shot2026.06 | 26 | 32.54 | — | |
| Direct promptingModel=Davinci-0022022.10 | 25.4 | — | — | |
| SearchModel=Davinci-0022022.10 | 2.2 | — | — |