# Logical Reasoning on RP LP Domain, Npred ≤ 60, 6 \< δ ≤ 12 Source: [Logical Reasoning on RP LP Domain, Npred ≤ 60, 6 \< δ ≤ 12]() Collection index: [llms.txt]() Data updated: 2026-05-07T06:55:46.394Z Evaluation of model performance on Logical Reasoning using the RP LP Domain subset constrained by Npred ≤ 60 and 6 \< δ ≤ 12, measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Logical Reasoning | | Dataset (including any stated split) | RP LP Domain, Npred ≤ 60, 6 \< δ ≤ 12 | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | cot w. L = 64 | 93.5 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 128 | 91.8 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 128 | 83 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 64 | 81.6 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 128 | 79.8 | Not recorded | 69fc1c725497b3b670c88433 | | cot (baseline) | 79.7 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 128 | 78 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 96 | 77.3 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 48 | 75.2 | Not recorded | 69fc1c725497b3b670c88433 | | direct (baseline) | 73.5 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 64 | 71.9 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 32 | 71.3 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 24 | 70.1 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 96 | 66.2 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 48 | 63.6 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. Nepochs = 60 | 62.5 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. L = 16 | 61 | Not recorded | 69fc1c725497b3b670c88433 | | cot (baseline) | 60.8 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 24 | 60.3 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 32 | 59.7 | Not recorded | 69fc1c725497b3b670c88433 | | cot w. Nepochs = 30 | 58.3 | Not recorded | 69fc1c725497b3b670c88433 | | direct (baseline) | 55.4 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. Nepochs = 60 | 54.8 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. L = 16 | 53.7 | Not recorded | 69fc1c725497b3b670c88433 | | direct w. Nepochs = 30 | 53.3 | Not recorded | 69fc1c725497b3b670c88433 | ### cot w. L = 64 Accuracy: 93.5. Result ID: 69fc368947f34ce924de8353. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"64"}. This record does not assert independent reproduction. ### cot w. L = 128 Accuracy: 91.8. Result ID: 69fc368947f34ce924de8354. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"128"}. This record does not assert independent reproduction. ### cot w. L = 128 Accuracy: 83. Result ID: 69fc368947f34ce924de833f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"128"}. This record does not assert independent reproduction. ### cot w. L = 64 Accuracy: 81.6. Result ID: 69fc368947f34ce924de833d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"64"}. This record does not assert independent reproduction. ### direct w. L = 128 Accuracy: 79.8. Result ID: 69fc368947f34ce924de8351. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"128"}. This record does not assert independent reproduction. ### cot (baseline) Accuracy: 79.7. Result ID: 69fc368947f34ce924de8352. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT"}. This record does not assert independent reproduction. ### direct w. L = 128 Accuracy: 78. Result ID: 69fc368947f34ce924de8335. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"128"}. This record does not assert independent reproduction. ### cot w. L = 96 Accuracy: 77.3. Result ID: 69fc368947f34ce924de833e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"96"}. This record does not assert independent reproduction. ### cot w. L = 48 Accuracy: 75.2. Result ID: 69fc368947f34ce924de833c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"48"}. This record does not assert independent reproduction. ### direct (baseline) Accuracy: 73.5. Result ID: 69fc368947f34ce924de8350. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct"}. This record does not assert independent reproduction. ### direct w. L = 64 Accuracy: 71.9. Result ID: 69fc368947f34ce924de8333. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"64"}. This record does not assert independent reproduction. ### cot w. L = 32 Accuracy: 71.3. Result ID: 69fc368947f34ce924de833b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"32"}. This record does not assert independent reproduction. ### cot w. L = 24 Accuracy: 70.1. Result ID: 69fc368947f34ce924de833a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"24"}. This record does not assert independent reproduction. ### direct w. L = 96 Accuracy: 66.2. Result ID: 69fc368947f34ce924de8334. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"96"}. This record does not assert independent reproduction. ### direct w. L = 48 Accuracy: 63.6. Result ID: 69fc368947f34ce924de8332. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"48"}. This record does not assert independent reproduction. ### cot w. Nepochs = 60 Accuracy: 62.5. Result ID: 69fc368947f34ce924de8338. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","N\_epochs":"60"}. This record does not assert independent reproduction. ### cot w. L = 16 Accuracy: 61. Result ID: 69fc368947f34ce924de8339. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","Number of Layers":"16"}. This record does not assert independent reproduction. ### cot (baseline) Accuracy: 60.8. Result ID: 69fc368947f34ce924de8336. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT"}. This record does not assert independent reproduction. ### direct w. L = 24 Accuracy: 60.3. Result ID: 69fc368947f34ce924de8330. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"24"}. This record does not assert independent reproduction. ### direct w. L = 32 Accuracy: 59.7. Result ID: 69fc368947f34ce924de8331. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"32"}. This record does not assert independent reproduction. ### cot w. Nepochs = 30 Accuracy: 58.3. Result ID: 69fc368947f34ce924de8337. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"CoT","N\_epochs":"30"}. This record does not assert independent reproduction. ### direct (baseline) Accuracy: 55.4. Result ID: 69fc368947f34ce924de832c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct"}. This record does not assert independent reproduction. ### direct w. Nepochs = 60 Accuracy: 54.8. Result ID: 69fc368947f34ce924de832e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","N\_epochs":"60"}. This record does not assert independent reproduction. ### direct w. L = 16 Accuracy: 53.7. Result ID: 69fc368947f34ce924de832f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","Number of Layers":"16"}. This record does not assert independent reproduction. ### direct w. Nepochs = 30 Accuracy: 53.3. Result ID: 69fc368947f34ce924de832d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reasoning Strategy":"Direct","N\_epochs":"30"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.