# Multi-hop Reasoning on MuSiQue IRCoT 500 samples (test) Source: [Multi-hop Reasoning on MuSiQue IRCoT 500 samples (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:21:49.475Z Evaluation of multi-hop reasoning performance on the 500-sample test split of the MuSiQue dataset using the IRCoT setup, measured by ACC and F1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-hop Reasoning | | Dataset (including any stated split) | MuSiQue IRCoT 500 samples (test) | | Metric | ACC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ACC](): Higher is better; unit: PERCENT. - [F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | ACC | Result date | Report paper | | --- | --- | --- | --- | | ActiShade | 25.59 | Not recorded | 6965d1996343fa78c8323669 | | ActiShade | 25.25 | Not recorded | 6965d1996343fa78c8323669 | | ActiShade | 22.8 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 22.7 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 21.11 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 20.6 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 19.8 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 19.74 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 18.48 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 18.24 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 17.35 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 17.2 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 15.89 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 15.57 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 15.4 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 14.9 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 14.8 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 13.1 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 11.65 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 11.6 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 11.42 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 10.69 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 10.4 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 6.2 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 6 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 5.6 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 3.8 | Not recorded | 6965d1996343fa78c8323669 | ### ActiShade ACC: 25.59. Result ID: 69968edd5698665ec4d7ba81. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### ActiShade ACC: 25.25. Result ID: 69968edd5698665ec4d7ba6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### ActiShade ACC: 22.8. Result ID: 69968edd5698665ec4d7ba78. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 22.7. Result ID: 69968edd5698665ec4d7ba80. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 21.11. Result ID: 69968edd5698665ec4d7ba6e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 20.6. Result ID: 69968edd5698665ec4d7ba6c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 19.8. Result ID: 69968edd5698665ec4d7ba77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 19.74. Result ID: 69968edd5698665ec4d7ba6d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 18.48. Result ID: 69968edd5698665ec4d7ba7e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 18.24. Result ID: 69968edd5698665ec4d7ba6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 17.35. Result ID: 69968edd5698665ec4d7ba75. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 17.2. Result ID: 69968edd5698665ec4d7ba7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 15.89. Result ID: 69968edd5698665ec4d7ba7d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 15.57. Result ID: 69968edd5698665ec4d7ba6b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 15.4. Result ID: 69968edd5698665ec4d7ba73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 14.9. Result ID: 69968edd5698665ec4d7ba74. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 14.8. Result ID: 69968edd5698665ec4d7ba7b. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 13.1. Result ID: 69968edd5698665ec4d7ba7f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### CoT ACC: 11.65. Result ID: 69968edd5698665ec4d7ba68. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct-R ACC: 11.6. Result ID: 69968edd5698665ec4d7ba72. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 11.42. Result ID: 69968edd5698665ec4d7ba69. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 10.69. Result ID: 69968edd5698665ec4d7ba76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### CoT ACC: 10.4. Result ID: 69968edd5698665ec4d7ba7a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 6.2. Result ID: 69968edd5698665ec4d7ba79. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### CoT ACC: 6. Result ID: 69968edd5698665ec4d7ba71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 5.6. Result ID: 69968edd5698665ec4d7ba67. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 3.8. Result ID: 69968edd5698665ec4d7ba70. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.