# Physical Reasoning on Kinetix Source: [Physical Reasoning on Kinetix]() Collection index: [llms.txt]() Data updated: 2026-03-17T07:30:03.250Z Evaluation of model performance on the Physical Reasoning task using the Kinetix dataset, measured by Att. 1 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Physical Reasoning | | Dataset (including any stated split) | Kinetix | | Metric | Att. 1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Att. 1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Att. 1 Score | Result date | Report paper | | --- | --- | --- | --- | | GPT-o3 | 26.89 | Not recorded | 69b8e0060636e5bf4afbe99b | | GPT-o3 | 24.5 | Not recorded | 69b8e0060636e5bf4afbe99b | | Gemini-2.5-Pro | 24.1 | Not recorded | 69b8e0060636e5bf4afbe99b | | Claude 4.0 Opus | 23.2 | Not recorded | 69b8e0060636e5bf4afbe99b | | Gemini-2.5-Pro | 21.8 | Not recorded | 69b8e0060636e5bf4afbe99b | | MOCK | 21.4 | Not recorded | 69b8e0060636e5bf4afbe99b | | Claude 4.0 Opus | 20.5 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P&I GRPO | 19.45 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P&I GRPO | 19.18 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P&I GRPO | 19.18 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P&I GRPO | 18.96 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^I GRPO | 16.44 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^I GRPO | 16.44 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B | 16.22 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P SFT | 15.31 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-32B | 15.2 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P&I SFT | 15.11 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P&I SFT | 15.07 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P SFT | 15.07 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P&I SFT | 15.07 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P&I SFT | 14.88 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-72B | 14.86 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P GRPO | 13.91 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P SFT | 13.7 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^I SFT | 13.7 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^P GRPO | 13.7 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P SFT | 13.55 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B | 13.51 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-7B^I SFT | 13.49 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P GRPO | 12.33 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^P GRPO | 12.33 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-72B | 12.16 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^I SFT | 9.82 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^I SFT | 9.6 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^I GRPO | 8.2 | Not recorded | 69b8e0060636e5bf4afbe99b | | Qwen-3B^I GRPO | 8.07 | Not recorded | 69b8e0060636e5bf4afbe99b | ### GPT-o3 Att. 1 Score: 26.89. Result ID: 69b902da47f34ce924a12297. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"GPT-o3","World Model":"-"}. This record does not assert independent reproduction. ### GPT-o3 Att. 1 Score: 24.5. Result ID: 69b902da47f34ce924a12298. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"GPT-o3","World Model":"GPT-o3"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Att. 1 Score: 24.1. Result ID: 69b902da47f34ce924a12295. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Gemini-2.5-Pro","World Model":"-"}. This record does not assert independent reproduction. ### Claude 4.0 Opus Att. 1 Score: 23.2. Result ID: 69b902da47f34ce924a12293. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Claude 4.0 Opus","World Model":"-"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Att. 1 Score: 21.8. Result ID: 69b902da47f34ce924a12296. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Gemini-2.5-Pro","World Model":"Gemini-2.5-Pro"}. This record does not assert independent reproduction. ### MOCK Att. 1 Score: 21.4. Result ID: 69b902da47f34ce924a1228d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"MOCK","World Model":"-"}. This record does not assert independent reproduction. ### Claude 4.0 Opus Att. 1 Score: 20.5. Result ID: 69b902da47f34ce924a12294. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Claude 4.0 Opus","World Model":"Claude 4.0 Opus"}. This record does not assert independent reproduction. ### Qwen-3B^P&I GRPO Att. 1 Score: 19.45. Result ID: 69b902da47f34ce924a122a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P&I GRPO","World Model":"Qwen-3B^P&I"}. This record does not assert independent reproduction. ### Qwen-3B^P&I GRPO Att. 1 Score: 19.18. Result ID: 69b902da47f34ce924a122a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P&I GRPO","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P&I GRPO Att. 1 Score: 19.18. Result ID: 69b902da47f34ce924a122af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P&I GRPO","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P&I GRPO Att. 1 Score: 18.96. Result ID: 69b902da47f34ce924a122b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P&I GRPO","World Model":"Qwen-7B^P&I"}. This record does not assert independent reproduction. ### Qwen-7B^I GRPO Att. 1 Score: 16.44. Result ID: 69b902da47f34ce924a122ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^I GRPO","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^I GRPO Att. 1 Score: 16.44. Result ID: 69b902da47f34ce924a122ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^I GRPO","World Model":"Qwen-7B^I"}. This record does not assert independent reproduction. ### Qwen-3B Att. 1 Score: 16.22. Result ID: 69b902da47f34ce924a1228e. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P SFT Att. 1 Score: 15.31. Result ID: 69b902da47f34ce924a122a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P SFT","World Model":"Qwen-7B^P"}. This record does not assert independent reproduction. ### Qwen-32B Att. 1 Score: 15.2. Result ID: 69b902da47f34ce924a12290. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-32B","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-3B^P&I SFT Att. 1 Score: 15.11. Result ID: 69b902da47f34ce924a1229e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P&I SFT","World Model":"Qwen-3B^P&I"}. This record does not assert independent reproduction. ### Qwen-3B^P&I SFT Att. 1 Score: 15.07. Result ID: 69b902da47f34ce924a1229d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P&I SFT","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P SFT Att. 1 Score: 15.07. Result ID: 69b902da47f34ce924a122a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P SFT","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P&I SFT Att. 1 Score: 15.07. Result ID: 69b902da47f34ce924a122a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P&I SFT","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P&I SFT Att. 1 Score: 14.88. Result ID: 69b902da47f34ce924a122aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P&I SFT","World Model":"Qwen-7B^P&I"}. This record does not assert independent reproduction. ### Qwen-72B Att. 1 Score: 14.86. Result ID: 69b902da47f34ce924a12291. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-72B","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P GRPO Att. 1 Score: 13.91. Result ID: 69b902da47f34ce924a122ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P GRPO","World Model":"Qwen-7B^P"}. This record does not assert independent reproduction. ### Qwen-3B^P SFT Att. 1 Score: 13.7. Result ID: 69b902da47f34ce924a12299. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P SFT","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^I SFT Att. 1 Score: 13.7. Result ID: 69b902da47f34ce924a122a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^I SFT","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^P GRPO Att. 1 Score: 13.7. Result ID: 69b902da47f34ce924a122ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^P GRPO","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-3B^P SFT Att. 1 Score: 13.55. Result ID: 69b902da47f34ce924a1229a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P SFT","World Model":"Qwen-3B^P"}. This record does not assert independent reproduction. ### Qwen-7B Att. 1 Score: 13.51. Result ID: 69b902da47f34ce924a1228f. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-7B^I SFT Att. 1 Score: 13.49. Result ID: 69b902da47f34ce924a122a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-7B^I SFT","World Model":"Qwen-7B^I"}. This record does not assert independent reproduction. ### Qwen-3B^P GRPO Att. 1 Score: 12.33. Result ID: 69b902da47f34ce924a1229f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P GRPO","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-3B^P GRPO Att. 1 Score: 12.33. Result ID: 69b902da47f34ce924a122a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^P GRPO","World Model":"Qwen-3B^P"}. This record does not assert independent reproduction. ### Qwen-72B Att. 1 Score: 12.16. Result ID: 69b902da47f34ce924a12292. [Reporting paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-72B","World Model":"Qwen-72B"}. This record does not assert independent reproduction. ### Qwen-3B^I SFT Att. 1 Score: 9.82. Result ID: 69b902da47f34ce924a1229c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^I SFT","World Model":"Qwen-3B^I"}. This record does not assert independent reproduction. ### Qwen-3B^I SFT Att. 1 Score: 9.6. Result ID: 69b902da47f34ce924a1229b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^I SFT","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-3B^I GRPO Att. 1 Score: 8.2. Result ID: 69b902da47f34ce924a122a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^I GRPO","World Model":"-"}. This record does not assert independent reproduction. ### Qwen-3B^I GRPO Att. 1 Score: 8.07. Result ID: 69b902da47f34ce924a122a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy Model":"Qwen-3B^I GRPO","World Model":"Qwen-3B^I"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.