# Spatial Reasoning on 3D FORCE SAG 1.0 (test) Source: [Spatial Reasoning on 3D FORCE SAG 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-06-23T11:15:04.534Z Evaluation of model performance on the 3D FORCE SAG 1.0 (test) dataset for Spatial Reasoning tasks, measured by SAG and Average scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Spatial Reasoning | | Dataset (including any stated split) | 3D FORCE SAG 1.0 (test) | | Metric | SAG Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SAG Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | SAG Score | Result date | Report paper | | --- | --- | --- | --- | | SATURN | 94.4 | Not recorded | 6a3a1363206542251d7c7413 | | SATURN | 87.57 | Not recorded | 6a3a1363206542251d7c7413 | | Gemini-3.1-Pro | 75.3 | Not recorded | 6a3a1363206542251d7c7413 | | Qwen3.5-9B | 74.78 | Not recorded | 6a3a1363206542251d7c7413 | | Qwen3-VL-235B | 73.22 | Not recorded | 6a3a1363206542251d7c7413 | | GPT-5.1 | 71.39 | Not recorded | 6a3a1363206542251d7c7413 | | Qwen3-VL-8B | 64.17 | Not recorded | 6a3a1363206542251d7c7413 | | Qwen3.5-35B | 58.52 | Not recorded | 6a3a1363206542251d7c7413 | | InternVL3.5-38B | 58.17 | Not recorded | 6a3a1363206542251d7c7413 | | Molmo2-8B | 54.61 | Not recorded | 6a3a1363206542251d7c7413 | | InternVL3.5-8B | 53.3 | Not recorded | 6a3a1363206542251d7c7413 | | GCA | 51 | Not recorded | 6a3a1363206542251d7c7413 | | Random choice | 50.17 | Not recorded | 6a3a1363206542251d7c7413 | | SpaceOm-3B | 49.39 | Not recorded | 6a3a1363206542251d7c7413 | | SpaceThinker-3B | 47.91 | Not recorded | 6a3a1363206542251d7c7413 | | SpaceQwen-3B | 37.74 | Not recorded | 6a3a1363206542251d7c7413 | | Cosmos-Reason1-7B | 36.87 | Not recorded | 6a3a1363206542251d7c7413 | ### SATURN SAG Score: 94.4. Result ID: 6a3a6aa4fbc373592c379aac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Oracle 3D":"true"}. This record does not assert independent reproduction. ### SATURN SAG Score: 87.57. Result ID: 6a3a6aa4fbc373592c379aab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B-Instruct"}. This record does not assert independent reproduction. ### Gemini-3.1-Pro SAG Score: 75.3. Result ID: 6a3a6aa4fbc373592c379aa5. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3.5-9B SAG Score: 74.78. Result ID: 6a3a6aa4fbc373592c379a9f. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"9B"}. This record does not assert independent reproduction. ### Qwen3-VL-235B SAG Score: 73.22. Result ID: 6a3a6aa4fbc373592c379a9e. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"235B"}. This record does not assert independent reproduction. ### GPT-5.1 SAG Score: 71.39. Result ID: 6a3a6aa4fbc373592c379aa4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-VL-8B SAG Score: 64.17. Result ID: 6a3a6aa4fbc373592c379a9d. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### Qwen3.5-35B SAG Score: 58.52. Result ID: 6a3a6aa4fbc373592c379aa0. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"35B"}. This record does not assert independent reproduction. ### InternVL3.5-38B SAG Score: 58.17. Result ID: 6a3a6aa4fbc373592c379aa2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"38B"}. This record does not assert independent reproduction. ### Molmo2-8B SAG Score: 54.61. Result ID: 6a3a6aa4fbc373592c379aa3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### InternVL3.5-8B SAG Score: 53.3. Result ID: 6a3a6aa4fbc373592c379aa1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### GCA SAG Score: 51. Result ID: 6a3a6aa4fbc373592c379aaa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"type":"tool-augmented"}. This record does not assert independent reproduction. ### Random choice SAG Score: 50.17. Result ID: 6a3a6aa4fbc373592c379a9c. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### SpaceOm-3B SAG Score: 49.39. Result ID: 6a3a6aa4fbc373592c379aa7. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B"}. This record does not assert independent reproduction. ### SpaceThinker-3B SAG Score: 47.91. Result ID: 6a3a6aa4fbc373592c379aa8. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B"}. This record does not assert independent reproduction. ### SpaceQwen-3B SAG Score: 37.74. Result ID: 6a3a6aa4fbc373592c379aa6. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B"}. This record does not assert independent reproduction. ### Cosmos-Reason1-7B SAG Score: 36.87. Result ID: 6a3a6aa4fbc373592c379aa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.