# Multi-hop Reasoning on 2WikiMQA IRCoT 500 samples (test) Source: [Multi-hop Reasoning on 2WikiMQA IRCoT 500 samples (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:21:49.520Z Evaluation of multi-hop reasoning performance on the specific 500-sample test split of the 2WikiMQA dataset configured using the IRCoT prompting method, measured by ACC and F1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-hop Reasoning | | Dataset (including any stated split) | 2WikiMQA IRCoT 500 samples (test) | | Metric | ACC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ACC](): Higher is better; unit: PERCENT. - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | ACC | Result date | Report paper | | --- | --- | --- | --- | | ActiShade | 52.8 | Not recorded | 6965d1996343fa78c8323669 | | ActiShade | 51.13 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 48.1 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 47.49 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 45.9 | Not recorded | 6965d1996343fa78c8323669 | | ActiShade | 45.8 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 44.6 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 43.2 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 43.17 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 41.35 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 41.2 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 40.78 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 40.21 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 40.01 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 39.5 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 39.21 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 39 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 38.71 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 38.6 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 34.2 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 33.4 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 29 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 29 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 28.37 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 27.6 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 26.8 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 26.6 | Not recorded | 6965d1996343fa78c8323669 | ### ActiShade ACC: 52.8. Result ID: 69968edd5698665ec4d7baae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### ActiShade ACC: 51.13. Result ID: 69968edd5698665ec4d7bab7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 48.1. Result ID: 69968edd5698665ec4d7bab6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 47.49. Result ID: 69968edd5698665ec4d7bab4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 45.9. Result ID: 69968edd5698665ec4d7baad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### ActiShade ACC: 45.8. Result ID: 69968edd5698665ec4d7baa5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 44.6. Result ID: 69968edd5698665ec4d7bab3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 43.2. Result ID: 69968edd5698665ec4d7bab2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 43.17. Result ID: 69968edd5698665ec4d7baab. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 41.35. Result ID: 69968edd5698665ec4d7baa3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 41.2. Result ID: 69968edd5698665ec4d7baa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 40.78. Result ID: 69968edd5698665ec4d7baa4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 40.21. Result ID: 69968edd5698665ec4d7baaa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 40.01. Result ID: 69968edd5698665ec4d7bab5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 39.5. Result ID: 69968edd5698665ec4d7baa2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 39.21. Result ID: 69968edd5698665ec4d7baac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 39. Result ID: 69968edd5698665ec4d7bab1. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 38.71. Result ID: 69968edd5698665ec4d7baa0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 38.6. Result ID: 69968edd5698665ec4d7baa8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 34.2. Result ID: 69968edd5698665ec4d7baa1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### CoT ACC: 33.4. Result ID: 69968edd5698665ec4d7bab0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### CoT ACC: 29. Result ID: 69968edd5698665ec4d7baa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 29. Result ID: 69968edd5698665ec4d7baaf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct-R ACC: 28.37. Result ID: 69968edd5698665ec4d7ba9f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### CoT ACC: 27.6. Result ID: 69968edd5698665ec4d7ba9e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 26.8. Result ID: 69968edd5698665ec4d7baa6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 26.6. Result ID: 69968edd5698665ec4d7ba9d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.