# Multi-hop Reasoning on HotpotQA IRCoT (500 samples) (test) Source: [Multi-hop Reasoning on HotpotQA IRCoT (500 samples) (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:21:49.499Z Evaluation of model performance on a specific 500-sample subset of the HotpotQA dataset utilizing the IRCoT methodology for multi-hop reasoning, measured using ACC and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-hop Reasoning | | Dataset (including any stated split) | HotpotQA IRCoT (500 samples) (test) | | Metric | ACC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ACC](): Higher is better; unit: PERCENT. - [F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | ACC | Result date | Report paper | | --- | --- | --- | --- | | ActiShade | 54.6 | Not recorded | 6965d1996343fa78c8323669 | | ActiShade | 53.97 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 51.21 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 50.87 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 49.03 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 48.45 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 48.23 | Not recorded | 6965d1996343fa78c8323669 | | ActiShade | 48.2 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 47.98 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 47.1 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 46.2 | Not recorded | 6965d1996343fa78c8323669 | | DRAGIN | 46.1 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 45.97 | Not recorded | 6965d1996343fa78c8323669 | | Iter-RetGen | 44.4 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 43.79 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 43 | Not recorded | 6965d1996343fa78c8323669 | | SelfASK | 42.18 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 42.14 | Not recorded | 6965d1996343fa78c8323669 | | FLARE | 40.19 | Not recorded | 6965d1996343fa78c8323669 | | IRCoT | 40.1 | Not recorded | 6965d1996343fa78c8323669 | | Direct-R | 37.7 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 29.4 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 29 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 27.2 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 22.4 | Not recorded | 6965d1996343fa78c8323669 | | CoT | 22 | Not recorded | 6965d1996343fa78c8323669 | | Direct | 19.4 | Not recorded | 6965d1996343fa78c8323669 | ### ActiShade ACC: 54.6. Result ID: 69968edd5698665ec4d7ba8a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### ActiShade ACC: 53.97. Result ID: 69968edd5698665ec4d7ba9c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 51.21. Result ID: 69968edd5698665ec4d7ba9b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 50.87. Result ID: 69968edd5698665ec4d7ba89. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 49.03. Result ID: 69968edd5698665ec4d7ba97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 48.45. Result ID: 69968edd5698665ec4d7ba88. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 48.23. Result ID: 69968edd5698665ec4d7ba85. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### ActiShade ACC: 48.2. Result ID: 69968edd5698665ec4d7ba93. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Active/Iterative retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 47.98. Result ID: 69968edd5698665ec4d7ba98. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 47.1. Result ID: 69968edd5698665ec4d7ba87. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 46.2. Result ID: 69968edd5698665ec4d7ba96. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### DRAGIN ACC: 46.1. Result ID: 69968edd5698665ec4d7ba92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 45.97. Result ID: 69968edd5698665ec4d7ba99. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Iter-RetGen ACC: 44.4. Result ID: 69968edd5698665ec4d7ba8e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 43.79. Result ID: 69968edd5698665ec4d7ba8f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 43. Result ID: 69968edd5698665ec4d7ba8d. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### SelfASK ACC: 42.18. Result ID: 69968edd5698665ec4d7ba90. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 42.14. Result ID: 69968edd5698665ec4d7ba9a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### FLARE ACC: 40.19. Result ID: 69968edd5698665ec4d7ba91. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### IRCoT ACC: 40.1. Result ID: 69968edd5698665ec4d7ba86. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Multi-round retrieval"}. This record does not assert independent reproduction. ### Direct-R ACC: 37.7. Result ID: 69968edd5698665ec4d7ba84. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"Single-round retrieval"}. This record does not assert independent reproduction. ### CoT ACC: 29.4. Result ID: 69968edd5698665ec4d7ba95. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### CoT ACC: 29. Result ID: 69968edd5698665ec4d7ba83. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 27.2. Result ID: 69968edd5698665ec4d7ba94. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 22.4. Result ID: 69968edd5698665ec4d7ba82. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3-8B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### CoT ACC: 22. Result ID: 69968edd5698665ec4d7ba8c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. ### Direct ACC: 19.4. Result ID: 69968edd5698665ec4d7ba8b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","Retrieval Mode":"None"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.