# Mathematical Reasoning on OMEGA Source: [Mathematical Reasoning on OMEGA]() Collection index: [llms.txt]() Data updated: 2026-02-27T10:43:55.847Z Evaluation of model performance on the OMEGA dataset for mathematical reasoning tasks, measured by Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | OMEGA | | Metric | Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Score | Result date | Report paper | | --- | --- | --- | --- | | Olmo 3.1 Think 32B | 53.4 | Not recorded | 694238e845614693628503e0 | | Qwen 3 VL 32B Think | 50.8 | Not recorded | 694238e845614693628503e0 | | Olmo 3 Think (Final 3.0) | 50.6 | Not recorded | 694238e845614693628503e0 | | Qwen 3 32B | 47.7 | Not recorded | 694238e845614693628503e0 | | K2-V2 70B Instruct | 46.1 | Not recorded | 694238e845614693628503e0 | | Olmo 3 Think (DPO) | 45.2 | Not recorded | 694238e845614693628503e0 | | Olmo 3 7B Think | 45 | Not recorded | 694238e845614693628503e0 | | Qwen 3 8B | 43.4 | Not recorded | 694238e845614693628503e0 | | OR Nemotron 7B | 43.2 | Not recorded | 694238e845614693628503e0 | | Olmo 3 Think (SFT) | 43.1 | Not recorded | 694238e845614693628503e0 | | Nemotron Nano 9B v2 | 42.4 | Not recorded | 694238e845614693628503e0 | | Olmo 3 7B Think | 40.5 | Not recorded | 694238e845614693628503e0 | | DS-R1 32B | 38.9 | Not recorded | 694238e845614693628503e0 | | OpenThinker3 7B | 38.4 | Not recorded | 694238e845614693628503e0 | | Qwen 3 VL 8B Think | 38.1 | Not recorded | 694238e845614693628503e0 | | Olmo 3 7B Think | 37.8 | Not recorded | 694238e845614693628503e0 | | Qwen 3 VL 8B Inst | 32.3 | Not recorded | 694238e845614693628503e0 | | Olmo 3 7B Instruct | 28.9 | Not recorded | 694238e845614693628503e0 | | DS-R1 Qwen 7B | 28.5 | Not recorded | 694238e845614693628503e0 | | Olmo 3 7B Instruct | 22.8 | Not recorded | 694238e845614693628503e0 | | Qwen 3 8B | 20.5 | Not recorded | 694238e845614693628503e0 | | Gemma 3 12B | 15.19 | Not recorded | 698181a5a8b01e76cc8e2766 | | DictaLM 3.0 12B-Inst | 15.19 | Not recorded | 698181a5a8b01e76cc8e2766 | | Olmo 3 7B Instruct | 14.4 | Not recorded | 694238e845614693628503e0 | | Qwen 2.5 7B | 13.7 | Not recorded | 694238e845614693628503e0 | | Granite 3.3 8B Inst | 10.7 | Not recorded | 694238e845614693628503e0 | | OLMo 2 7B Inst | 5.2 | Not recorded | 694238e845614693628503e0 | | Apertus 8B Inst | 5 | Not recorded | 694238e845614693628503e0 | ### Olmo 3.1 Think 32B Score: 53.4. Result ID: 6996bf865698665ec40a944a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Stage":"Final Think 3.1","Model Family":"Olmo 3.1","Parameter Count":"32B","Thinking Capability":"true"}. This record does not assert independent reproduction. ### Qwen 3 VL 32B Think Score: 50.8. Result ID: 6996bf865698665ec40a944c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"Qwen 3","Parameter Count":"32B","Thinking Capability":"true"}. This record does not assert independent reproduction. ### Olmo 3 Think (Final 3.0) Score: 50.6. Result ID: 6996bf865698665ec40a9449. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Stage":"Final Think 3.0","Model Family":"Olmo 3","Parameter Count":"32B","Thinking Capability":"true"}. This record does not assert independent reproduction. ### Qwen 3 32B Score: 47.7. Result ID: 6996bf865698665ec40a944b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"Qwen 3","Parameter Count":"32B","Thinking Capability":"false"}. This record does not assert independent reproduction. ### K2-V2 70B Instruct Score: 46.1. Result ID: 6996bf865698665ec40a944e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"K2","Parameter Count":"70B","Thinking Capability":"false"}. This record does not assert independent reproduction. ### Olmo 3 Think (DPO) Score: 45.2. Result ID: 6996bf865698665ec40a9448. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Stage":"DPO","Model Family":"Olmo 3","Parameter Count":"32B","Thinking Capability":"true"}. This record does not assert independent reproduction. ### Olmo 3 7B Think Score: 45. Result ID: 6996bf865698665ec40a94c2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Stage":"Final Think"}. This record does not assert independent reproduction. ### Qwen 3 8B Score: 43.4. Result ID: 6996bf865698665ec40a94c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OR Nemotron 7B Score: 43.2. Result ID: 6996bf865698665ec40a94c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Olmo 3 Think (SFT) Score: 43.1. Result ID: 6996bf865698665ec40a9447. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Stage":"SFT","Model Family":"Olmo 3","Parameter Count":"32B","Thinking Capability":"true"}. This record does not assert independent reproduction. ### Nemotron Nano 9B v2 Score: 42.4. Result ID: 6996bf865698665ec40a94c4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Olmo 3 7B Think Score: 40.5. Result ID: 6996bf865698665ec40a94c1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Stage":"DPO"}. This record does not assert independent reproduction. ### DS-R1 32B Score: 38.9. Result ID: 6996bf865698665ec40a944d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"DeepSeek-R1","Parameter Count":"32B","Thinking Capability":"true"}. This record does not assert independent reproduction. ### OpenThinker3 7B Score: 38.4. Result ID: 6996bf865698665ec40a94c3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen 3 VL 8B Think Score: 38.1. Result ID: 6996bf865698665ec40a94c7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Olmo 3 7B Think Score: 37.8. Result ID: 6996bf865698665ec40a94c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Stage":"SFT"}. This record does not assert independent reproduction. ### Qwen 3 VL 8B Inst Score: 32.3. Result ID: 6996bf865698665ec40a9696. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"Instruct"}. This record does not assert independent reproduction. ### Olmo 3 7B Instruct Score: 28.9. Result ID: 6996bf865698665ec40a9694. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"Final Instruct"}. This record does not assert independent reproduction. ### DS-R1 Qwen 7B Score: 28.5. Result ID: 6996bf865698665ec40a94c5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Olmo 3 7B Instruct Score: 22.8. Result ID: 6996bf865698665ec40a9693. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"DPO"}. This record does not assert independent reproduction. ### Qwen 3 8B Score: 20.5. Result ID: 6996bf865698665ec40a9695. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"Instruct"}. This record does not assert independent reproduction. ### Gemma 3 12B Score: 15.19. Result ID: 69a1756835dfcabb00e72c1a. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"12B"}. This record does not assert independent reproduction. ### DictaLM 3.0 12B-Inst Score: 15.19. Result ID: 69a1756835dfcabb00e72c1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"12B","Variant":"Instruct"}. This record does not assert independent reproduction. ### Olmo 3 7B Instruct Score: 14.4. Result ID: 6996bf865698665ec40a9692. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"SFT"}. This record does not assert independent reproduction. ### Qwen 2.5 7B Score: 13.7. Result ID: 6996bf865698665ec40a9697. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"Instruct"}. This record does not assert independent reproduction. ### Granite 3.3 8B Inst Score: 10.7. Result ID: 6996bf865698665ec40a969a. [Reporting paper]() Recorded evaluation configuration: {"stage":"Instruct"}. This record does not assert independent reproduction. ### OLMo 2 7B Inst Score: 5.2. Result ID: 6996bf865698665ec40a9698. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"Instruct"}. This record does not assert independent reproduction. ### Apertus 8B Inst Score: 5. Result ID: 6996bf865698665ec40a9699. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"stage":"Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.