# Multi-hop reasoning on MuSiQue Open Book (test) Source: [Multi-hop reasoning on MuSiQue Open Book (test)]() Collection index: [llms.txt]() Data updated: 2026-06-04T07:28:39.956Z Evaluation of model performance on the test split of the MuSiQue Open Book dataset for multi-hop reasoning tasks, measured using Accuracy and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-hop reasoning | | Dataset (including any stated split) | MuSiQue Open Book (test) | | Metric | Accuracy / F1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy / F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy / F1 | Result date | Report paper | | --- | --- | --- | --- | | DeCoRe (static) | 75.05 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 74.64 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 74.43 | Not recorded | 6a210686c10484bac23f010c | | ITI | 74.27 | Not recorded | 6a210686c10484bac23f010c | | AD | 74.27 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 74.27 | Not recorded | 6a210686c10484bac23f010c | | CAD | 74.02 | Not recorded | 6a210686c10484bac23f010c | | CAD | 73.02 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 72.49 | Not recorded | 6a210686c10484bac23f010c | | CD | 71.7 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 69.92 | Not recorded | 6a210686c10484bac23f010c | | AD | 69.92 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 69.84 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 69.76 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 69.01 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 68.68 | Not recorded | 6a210686c10484bac23f010c | | CAD | 68.64 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 68.56 | Not recorded | 6a210686c10484bac23f010c | | ITI | 68.14 | Not recorded | 6a210686c10484bac23f010c | | AD | 68.14 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 67.89 | Not recorded | 6a210686c10484bac23f010c | | CD | 66.61 | Not recorded | 6a210686c10484bac23f010c | | DeCoRe (static) | 61.23 | Not recorded | 6a210686c10484bac23f010c | | DoLa | 59.08 | Not recorded | 6a210686c10484bac23f010c | | Greedy | 58.83 | Not recorded | 6a210686c10484bac23f010c | | AD | 58.63 | Not recorded | 6a210686c10484bac23f010c | | DCO (Ours) | 58.26 | Not recorded | 6a210686c10484bac23f010c | | CAD | 57.88 | Not recorded | 6a210686c10484bac23f010c | | ITI | 45.84 | Not recorded | 6a210686c10484bac23f010c | | ITI | 38.31 | Not recorded | 6a210686c10484bac23f010c | ### DeCoRe (static) Accuracy / F1: 75.05. Result ID: 6a212925fbc373592c1c51ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy / F1: 74.64. Result ID: 6a212925fbc373592c1c51ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy / F1: 74.43. Result ID: 6a212925fbc373592c1c51a7. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### ITI Accuracy / F1: 74.27. Result ID: 6a212925fbc373592c1c51a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### AD Accuracy / F1: 74.27. Result ID: 6a212925fbc373592c1c51ac. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy / F1: 74.27. Result ID: 6a212925fbc373592c1c51ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### CAD Accuracy / F1: 74.02. Result ID: 6a212925fbc373592c1c51aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### CAD Accuracy / F1: 73.02. Result ID: 6a212925fbc373592c1c519a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy / F1: 72.49. Result ID: 6a212925fbc373592c1c519d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### CD Accuracy / F1: 71.7. Result ID: 6a212925fbc373592c1c51a9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy / F1: 69.92. Result ID: 6a212925fbc373592c1c519b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### AD Accuracy / F1: 69.92. Result ID: 6a212925fbc373592c1c519c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy / F1: 69.84. Result ID: 6a212925fbc373592c1c5198. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy / F1: 69.76. Result ID: 6a212925fbc373592c1c51a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy / F1: 69.01. Result ID: 6a212925fbc373592c1c519e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy / F1: 68.68. Result ID: 6a212925fbc373592c1c51a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### CAD Accuracy / F1: 68.64. Result ID: 6a212925fbc373592c1c51a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy / F1: 68.56. Result ID: 6a212925fbc373592c1c519f. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### ITI Accuracy / F1: 68.14. Result ID: 6a212925fbc373592c1c51a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### AD Accuracy / F1: 68.14. Result ID: 6a212925fbc373592c1c51a4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy / F1: 67.89. Result ID: 6a212925fbc373592c1c51a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### CD Accuracy / F1: 66.61. Result ID: 6a212925fbc373592c1c51a1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-70B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DeCoRe (static) Accuracy / F1: 61.23. Result ID: 6a212925fbc373592c1c5196. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DoLa Accuracy / F1: 59.08. Result ID: 6a212925fbc373592c1c5194. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### Greedy Accuracy / F1: 58.83. Result ID: 6a212925fbc373592c1c5191. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### AD Accuracy / F1: 58.63. Result ID: 6a212925fbc373592c1c5195. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### DCO (Ours) Accuracy / F1: 58.26. Result ID: 6a212925fbc373592c1c5197. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### CAD Accuracy / F1: 57.88. Result ID: 6a212925fbc373592c1c5193. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### ITI Accuracy / F1: 45.84. Result ID: 6a212925fbc373592c1c5192. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"Without CoT"}. This record does not assert independent reproduction. ### ITI Accuracy / F1: 38.31. Result ID: 6a212925fbc373592c1c5199. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-8B","Chain-of-Thought":"With CoT"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.