# Multi-hop reasoning on MuSiQue Closed Book (test) Source: [Multi-hop reasoning on MuSiQue Closed Book (test)]() Collection index: [llms.txt]() Data updated: 2026-06-04T07:28:39.922Z Evaluation of model performance on the test split of the MuSiQue Closed Book dataset for multi-hop reasoning tasks. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-hop reasoning | | Dataset (including any stated split) | MuSiQue Closed Book (test) | | Metric | Accuracy/F1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy/F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy/F1 | Result date | Report paper | | --- | --- | --- | --- | | DCO (Ours) | 21.14 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 20.6 | Not recorded | 6a210686c10484bac23f010c | | ITI | 20.44 | Not recorded | 6a210686c10484bac23f010c | | AD | 20.23 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 20.15 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 20.15 | Not recorded | 6a210686c10484bac23f010c | | CD | 17.17 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 14.94 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 14.69 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 14.61 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 14.56 | Not recorded | 6a210686c10484bac23f010c | | AD | 14.4 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 11.96 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 11.79 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 11.79 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 11.42 | Not recorded | 6a210686c10484bac23f010c | | AD | 11.38 | Not recorded | 6a210686c10484bac23f010c | | CD | 10.92 | Not recorded | 6a210686c10484bac23f010c | | ITI | 10.88 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 7.9 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 7.41 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 7.41 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 7.24 | Not recorded | 6a210686c10484bac23f010c | | AD | 6.99 | Not recorded | 6a210686c10484bac23f010c | | ITI | 4.18 | Not recorded | 6a210686c10484bac23f010c | | ITI | 4.01 | Not recorded | 6a210686c10484bac23f010c | ### DCO (Ours) Accuracy/F1: 21.14. Result ID: 6a212925fbc373592c1c5190. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy/F1: 20.6. Result ID: 6a212925fbc373592c1c518f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### ITI Accuracy/F1: 20.44. Result ID: 6a212925fbc373592c1c518b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### AD Accuracy/F1: 20.23. Result ID: 6a212925fbc373592c1c518e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy/F1: 20.15. Result ID: 6a212925fbc373592c1c518a. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy/F1: 20.15. Result ID: 6a212925fbc373592c1c518d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### CD Accuracy/F1: 17.17. Result ID: 6a212925fbc373592c1c518c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy/F1: 14.94. Result ID: 6a212925fbc373592c1c517f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy/F1: 14.69. Result ID: 6a212925fbc373592c1c5181. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy/F1: 14.61. Result ID: 6a212925fbc373592c1c517d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy/F1: 14.56. Result ID: 6a212925fbc373592c1c5182. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### AD Accuracy/F1: 14.4. Result ID: 6a212925fbc373592c1c5180. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy/F1: 11.96. Result ID: 6a212925fbc373592c1c5189. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy/F1: 11.79. Result ID: 6a212925fbc373592c1c5183. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy/F1: 11.79. Result ID: 6a212925fbc373592c1c5188. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy/F1: 11.42. Result ID: 6a212925fbc373592c1c5186. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### AD Accuracy/F1: 11.38. Result ID: 6a212925fbc373592c1c5187. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### CD Accuracy/F1: 10.92. Result ID: 6a212925fbc373592c1c5185. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### ITI Accuracy/F1: 10.88. Result ID: 6a212925fbc373592c1c5184. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy/F1: 7.9. Result ID: 6a212925fbc373592c1c517b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy/F1: 7.41. Result ID: 6a212925fbc373592c1c5177. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy/F1: 7.41. Result ID: 6a212925fbc373592c1c517c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy/F1: 7.24. Result ID: 6a212925fbc373592c1c5179. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### AD Accuracy/F1: 6.99. Result ID: 6a212925fbc373592c1c517a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### ITI Accuracy/F1: 4.18. Result ID: 6a212925fbc373592c1c517e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### ITI Accuracy/F1: 4.01. Result ID: 6a212925fbc373592c1c5178. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.