# Mathematical Reasoning on Aggregate (AMC23, AIME24, MATH-500, Minerva, Olympiad) Source: [Mathematical Reasoning on Aggregate (AMC23, AIME24, MATH-500, Minerva, Olympiad)]() Collection index: [llms.txt]() Data updated: 2026-05-11T09:40:22.578Z Evaluation of Mathematical Reasoning performance across an aggregate benchmark comprising AMC23, AIME24, MATH-500, Minerva, and Olympiad datasets, reporting accuracy deltas, token deltas, and Intelligence Per Token (IPT). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | Aggregate AMC23, AIME24, MATH-500, Minerva, Olympiad | | Metric | Intelligence Per Token (IPT) | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Intelligence Per Token (IPT)](): Higher is better; unit: SCALAR. - [Accuracy Delta (%)](): Higher is better; unit: PERCENT. - [Token Delta (%)](): Lower is better; unit: PERCENT. ## Recorded results | Method | Intelligence Per Token (IPT) | Result date | Report paper | | --- | --- | --- | --- | | TRiMS | 68.2 | Not recorded | 69bb83200636e5bf4afc1b3e | | ExpThink | 51.04 | Not recorded | 6a0162afdbc836b87d823244 | | TRiMS | 45.1 | Not recorded | 69bb83200636e5bf4afc1b3e | | ExpThink | 32.84 | Not recorded | 6a0162afdbc836b87d823244 | | LC-R1 | 28.4 | Not recorded | 69bb83200636e5bf4afc1b3e | | Laser | 27.8 | Not recorded | 69bb83200636e5bf4afc1b3e | | LC-R1 | 23.47 | Not recorded | 6a0162afdbc836b87d823244 | | Laser | 23.3 | Not recorded | 6a0162afdbc836b87d823244 | | ExpThink | 23.29 | Not recorded | 6a0162afdbc836b87d823244 | | JET | 21.1 | Not recorded | 69bb83200636e5bf4afc1b3e | | AdaptThink | 19.52 | Not recorded | 6a0162afdbc836b87d823244 | | AdaptThink | 19.3 | Not recorded | 69bb83200636e5bf4afc1b3e | | AutoThink | 18.63 | Not recorded | 6a0162afdbc836b87d823244 | | AdaptThink | 18.5 | Not recorded | 69bb83200636e5bf4afc1b3e | | AdaptThink | 18.04 | Not recorded | 6a0162afdbc836b87d823244 | | AutoThink | 17.9 | Not recorded | 69bb83200636e5bf4afc1b3e | | JET | 16.7 | Not recorded | 69bb83200636e5bf4afc1b3e | | Laser | 15.8 | Not recorded | 69bb83200636e5bf4afc1b3e | | AutoThink | 14.9 | Not recorded | 69bb83200636e5bf4afc1b3e | | LC-R1 | 14.49 | Not recorded | 6a0162afdbc836b87d823244 | | JET | 14.35 | Not recorded | 6a0162afdbc836b87d823244 | | LC-R1 | 13.6 | Not recorded | 69bb83200636e5bf4afc1b3e | | Laser | 13.35 | Not recorded | 6a0162afdbc836b87d823244 | | AutoThink | 13.25 | Not recorded | 6a0162afdbc836b87d823244 | | Vanilla | 11.2 | Not recorded | 69bb83200636e5bf4afc1b3e | | Vanilla | 11.15 | Not recorded | 6a0162afdbc836b87d823244 | | JET | 10.86 | Not recorded | 6a0162afdbc836b87d823244 | | Vanilla | 8.64 | Not recorded | 6a0162afdbc836b87d823244 | | Vanilla | 7.23 | Not recorded | 6a0162afdbc836b87d823244 | | Vanilla | 7.2 | Not recorded | 69bb83200636e5bf4afc1b3e | ### TRiMS Intelligence Per Token (IPT): 68.2. Result ID: 69bbbf8c47f34ce924a5e538. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### ExpThink Intelligence Per Token (IPT): 51.04. Result ID: 6a01a40447f34ce924e3e774. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### TRiMS Intelligence Per Token (IPT): 45.1. Result ID: 69bbbf8c47f34ce924a5e531. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. ### ExpThink Intelligence Per Token (IPT): 32.84. Result ID: 6a01a40447f34ce924e3e772. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### LC-R1 Intelligence Per Token (IPT): 28.4. Result ID: 69bbbf8c47f34ce924a5e533. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### Laser Intelligence Per Token (IPT): 27.8. Result ID: 69bbbf8c47f34ce924a5e534. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### LC-R1 Intelligence Per Token (IPT): 23.47. Result ID: 6a01a40447f34ce924e3e76d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### Laser Intelligence Per Token (IPT): 23.3. Result ID: 6a01a40447f34ce924e3e76e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### ExpThink Intelligence Per Token (IPT): 23.29. Result ID: 6a01a40447f34ce924e3e76b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### JET Intelligence Per Token (IPT): 21.1. Result ID: 69bbbf8c47f34ce924a5e537. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### AdaptThink Intelligence Per Token (IPT): 19.52. Result ID: 6a01a40447f34ce924e3e770. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### AdaptThink Intelligence Per Token (IPT): 19.3. Result ID: 69bbbf8c47f34ce924a5e536. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### AutoThink Intelligence Per Token (IPT): 18.63. Result ID: 6a01a40447f34ce924e3e76f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### AdaptThink Intelligence Per Token (IPT): 18.5. Result ID: 69bbbf8c47f34ce924a5e52f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. ### AdaptThink Intelligence Per Token (IPT): 18.04. Result ID: 6a01a40447f34ce924e3e769. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### AutoThink Intelligence Per Token (IPT): 17.9. Result ID: 69bbbf8c47f34ce924a5e535. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### JET Intelligence Per Token (IPT): 16.7. Result ID: 69bbbf8c47f34ce924a5e530. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. ### Laser Intelligence Per Token (IPT): 15.8. Result ID: 69bbbf8c47f34ce924a5e52d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. ### AutoThink Intelligence Per Token (IPT): 14.9. Result ID: 69bbbf8c47f34ce924a5e52e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. ### LC-R1 Intelligence Per Token (IPT): 14.49. Result ID: 6a01a40447f34ce924e3e766. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### JET Intelligence Per Token (IPT): 14.35. Result ID: 6a01a40447f34ce924e3e771. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### LC-R1 Intelligence Per Token (IPT): 13.6. Result ID: 69bbbf8c47f34ce924a5e52c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. ### Laser Intelligence Per Token (IPT): 13.35. Result ID: 6a01a40447f34ce924e3e767. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### AutoThink Intelligence Per Token (IPT): 13.25. Result ID: 6a01a40447f34ce924e3e768. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### Vanilla Intelligence Per Token (IPT): 11.2. Result ID: 69bbbf8c47f34ce924a5e532. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"7B"}. This record does not assert independent reproduction. ### Vanilla Intelligence Per Token (IPT): 11.15. Result ID: 6a01a40447f34ce924e3e76c. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### JET Intelligence Per Token (IPT): 10.86. Result ID: 6a01a40447f34ce924e3e76a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### Vanilla Intelligence Per Token (IPT): 8.64. Result ID: 6a01a40447f34ce924e3e773. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### Vanilla Intelligence Per Token (IPT): 7.23. Result ID: 6a01a40447f34ce924e3e765. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction. ### Vanilla Intelligence Per Token (IPT): 7.2. Result ID: 69bbbf8c47f34ce924a5e52b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen","Model Scale":"1.5B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.