# Logic Reasoning on K&K Logic Puzzles OOD Source: [Logic Reasoning on K&K Logic Puzzles OOD]() Collection index: [llms.txt]() Data updated: 2026-03-10T09:32:35.314Z Evaluation of model performance on the K&K Logic Puzzles dataset focusing on Out-of-Distribution (OOD) reasoning capabilities, measured by specific score thresholds (2 and 8) and overall average. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Logic Reasoning | | Dataset (including any stated split) | K&K Logic Puzzles OOD | | Metric | Score Threshold 2 (OOD) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score Threshold 2 (OOD)](): Higher is better; unit: PERCENT. - [Score Threshold 8 (OOD)](): Higher is better; unit: PERCENT. - [Average Score (OOD)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Score Threshold 2 (OOD) | Result date | Report paper | | --- | --- | --- | --- | | o3-mini-high | 99 | Not recorded | 68fa7b961e209131cea894b1 | | MeRF | 99 | Not recorded | 68fa7b961e209131cea894b1 | | MeRF | 99 | Not recorded | 68fa7b961e209131cea894b1 | | Deepseek-R1 | 91 | Not recorded | 68fa7b961e209131cea894b1 | | RLVR | 90 | Not recorded | 68fa7b961e209131cea894b1 | | RLVR | 90 | Not recorded | 68fa7b961e209131cea894b1 | | o1-2024-12-17 | 83 | Not recorded | 68fa7b961e209131cea894b1 | | MeRF | 76 | Not recorded | 68fa7b961e209131cea894b1 | | MeRF | 76 | Not recorded | 68fa7b961e209131cea894b1 | | RLVR | 71 | Not recorded | 68fa7b961e209131cea894b1 | | MeRF | 70 | Not recorded | 68fa7b961e209131cea894b1 | | GPT-4o | 68 | Not recorded | 68fa7b961e209131cea894b1 | | GPT-4o-mini | 63 | Not recorded | 68fa7b961e209131cea894b1 | | Qwen2.5-14B-Instruct | 63 | Not recorded | 68fa7b961e209131cea894b1 | | RLVR | 60 | Not recorded | 68fa7b961e209131cea894b1 | | RLVR | 59 | Not recorded | 68fa7b961e209131cea894b1 | | Qwen2.5-7B-Instruct-1M | 49 | Not recorded | 68fa7b961e209131cea894b1 | | Qwen2.5-7B-Instruct | 43 | Not recorded | 68fa7b961e209131cea894b1 | | MeRF | 43 | Not recorded | 68fa7b961e209131cea894b1 | | Qwen2.5-7B-Base | 41 | Not recorded | 68fa7b961e209131cea894b1 | | Deepseek-Math-7B | 35 | Not recorded | 68fa7b961e209131cea894b1 | | DeepSeek-R1-Distill-Qwen-1.5B | 30 | Not recorded | 68fa7b961e209131cea894b1 | | RLVR | 29 | Not recorded | 68fa7b961e209131cea894b1 | | NuminaMath-7B-CoT | 28 | Not recorded | 68fa7b961e209131cea894b1 | | DeepSeek-R1-Distill-Llama-8B | 24 | Not recorded | 68fa7b961e209131cea894b1 | ### o3-mini-high Score Threshold 2 (OOD): 99. Result ID: 69afe52b6dbb7075b9589e14. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MeRF Score Threshold 2 (OOD): 99. Result ID: 69afe52b6dbb7075b9589e26. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction. ### MeRF Score Threshold 2 (OOD): 99. Result ID: 69afe52b6dbb7075b9589e2c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Llama-8B"}. This record does not assert independent reproduction. ### Deepseek-R1 Score Threshold 2 (OOD): 91. Result ID: 69afe52b6dbb7075b9589e16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### RLVR Score Threshold 2 (OOD): 90. Result ID: 69afe52b6dbb7075b9589e25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction. ### RLVR Score Threshold 2 (OOD): 90. Result ID: 69afe52b6dbb7075b9589e2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Llama-8B"}. This record does not assert independent reproduction. ### o1-2024-12-17 Score Threshold 2 (OOD): 83. Result ID: 69afe52b6dbb7075b9589e15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MeRF Score Threshold 2 (OOD): 76. Result ID: 69afe52b6dbb7075b9589e20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### MeRF Score Threshold 2 (OOD): 76. Result ID: 69afe52b6dbb7075b9589e23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct-1M"}. This record does not assert independent reproduction. ### RLVR Score Threshold 2 (OOD): 71. Result ID: 69afe52b6dbb7075b9589e1f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### MeRF Score Threshold 2 (OOD): 70. Result ID: 69afe52b6dbb7075b9589e1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Base"}. This record does not assert independent reproduction. ### GPT-4o Score Threshold 2 (OOD): 68. Result ID: 69afe52b6dbb7075b9589e17. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GPT-4o-mini Score Threshold 2 (OOD): 63. Result ID: 69afe52b6dbb7075b9589e18. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2.5-14B-Instruct Score Threshold 2 (OOD): 63. Result ID: 69afe52b6dbb7075b9589e24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### RLVR Score Threshold 2 (OOD): 60. Result ID: 69afe52b6dbb7075b9589e22. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct-1M"}. This record does not assert independent reproduction. ### RLVR Score Threshold 2 (OOD): 59. Result ID: 69afe52b6dbb7075b9589e1c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Base"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct-1M Score Threshold 2 (OOD): 49. Result ID: 69afe52b6dbb7075b9589e21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Score Threshold 2 (OOD): 43. Result ID: 69afe52b6dbb7075b9589e1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MeRF Score Threshold 2 (OOD): 43. Result ID: 69afe52b6dbb7075b9589e29. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Base Score Threshold 2 (OOD): 41. Result ID: 69afe52b6dbb7075b9589e1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Deepseek-Math-7B Score Threshold 2 (OOD): 35. Result ID: 69afe52b6dbb7075b9589e1a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### DeepSeek-R1-Distill-Qwen-1.5B Score Threshold 2 (OOD): 30. Result ID: 69afe52b6dbb7075b9589e27. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### RLVR Score Threshold 2 (OOD): 29. Result ID: 69afe52b6dbb7075b9589e28. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### NuminaMath-7B-CoT Score Threshold 2 (OOD): 28. Result ID: 69afe52b6dbb7075b9589e19. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### DeepSeek-R1-Distill-Llama-8B Score Threshold 2 (OOD): 24. Result ID: 69afe52b6dbb7075b9589e2a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.