# Mathematical Reasoning on MATH 500 (Accuracy, Total Tokens) Source: [Mathematical Reasoning on MATH 500 (Accuracy, Total Tokens)]() Collection index: [llms.txt]() Data updated: 2026-05-11T10:03:04.733Z Evaluation of model performance on mathematical reasoning problems using the MATH 500 dataset, reporting standard Accuracy and inference cost measured by Total Tokens. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | MATH 500 | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Total Tokens](): Lower is better; unit: SCALAR. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | SC@5 + IRIS+ | 83.6 | Not recorded | 6a0162b8dbc836b87d8232fd | | IRIS+ | 78.8 | Not recorded | 6a0162b8dbc836b87d8232fd | | SC@5 nothink | 76.6 | Not recorded | 6a0162b8dbc836b87d8232fd | | IRIS | 74 | Not recorded | 6a0162b8dbc836b87d8232fd | | TOWN+ | 72.4 | Not recorded | 6a0162b8dbc836b87d8232fd | | TOWN | 71.8 | Not recorded | 6a0162b8dbc836b87d8232fd | | Think | 71 | Not recorded | 6a0162b8dbc836b87d8232fd | | Nothink | 68.6 | Not recorded | 6a0162b8dbc836b87d8232fd | | Nothink | 68.4 | Not recorded | 6a0162b8dbc836b87d8232fd | | IRIS | 67.2 | Not recorded | 6a0162b8dbc836b87d8232fd | | Nothink | 59.8 | Not recorded | 6a0162b8dbc836b87d8232fd | | SC@5 nothink | 57 | Not recorded | 6a0162b8dbc836b87d8232fd | | SC@3 nothink | 55.8 | Not recorded | 6a0162b8dbc836b87d8232fd | | TOWN | 55 | Not recorded | 6a0162b8dbc836b87d8232fd | | Think | 54.8 | Not recorded | 6a0162b8dbc836b87d8232fd | | Nothink | 47.5 | Not recorded | 6a0162b8dbc836b87d8232fd | ### SC@5 + IRIS+ Accuracy: 83.6. Result ID: 6a01a95847f34ce924e40f86. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"n":"500","Hardware":"H800","Extraction Strategy":"Fixed non-oracle agreement gate"}. This record does not assert independent reproduction. ### IRIS+ Accuracy: 78.8. Result ID: 6a01a95847f34ce924e40f8c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"4096","n":"500","Hardware":"H800","Extraction Strategy":"Strengthened extraction"}. This record does not assert independent reproduction. ### SC@5 nothink Accuracy: 76.6. Result ID: 6a01a95847f34ce924e40f85. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"1024","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### IRIS Accuracy: 74. Result ID: 6a01a95847f34ce924e40f8a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"4096","n":"500","Hardware":"H800","Extraction Strategy":"full"}. This record does not assert independent reproduction. ### TOWN+ Accuracy: 72.4. Result ID: 6a01a95847f34ce924e40f8b. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"4096","n":"500","Hardware":"H800","Extraction Strategy":"Strengthened extraction"}. This record does not assert independent reproduction. ### TOWN Accuracy: 71.8. Result ID: 6a01a95847f34ce924e40f89. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"4096","n":"500","Hardware":"H800","Extraction Strategy":"full"}. This record does not assert independent reproduction. ### Think Accuracy: 71. Result ID: 6a01a95847f34ce924e40f82. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"4096","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### Nothink Accuracy: 68.6. Result ID: 6a01a95847f34ce924e40f80. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"4096","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### Nothink Accuracy: 68.4. Result ID: 6a01a95847f34ce924e40f7f. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"2048","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### IRIS Accuracy: 67.2. Result ID: 6a01a95847f34ce924e40f88. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"2048","n":"500","Hardware":"H800","Extraction Strategy":"full"}. This record does not assert independent reproduction. ### Nothink Accuracy: 59.8. Result ID: 6a01a95847f34ce924e40f7e. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"1024","n":"500","Hardware":"A100"}. This record does not assert independent reproduction. ### SC@5 nothink Accuracy: 57. Result ID: 6a01a95847f34ce924e40f84. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"512","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### SC@3 nothink Accuracy: 55.8. Result ID: 6a01a95847f34ce924e40f83. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"512","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### TOWN Accuracy: 55. Result ID: 6a01a95847f34ce924e40f87. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"2048","n":"500","Hardware":"H800","Extraction Strategy":"full"}. This record does not assert independent reproduction. ### Think Accuracy: 54.8. Result ID: 6a01a95847f34ce924e40f81. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Budget (b)":"2048","n":"500","Hardware":"H800"}. This record does not assert independent reproduction. ### Nothink Accuracy: 47.5. Result ID: 6a01a95847f34ce924e40f7d. [Reporting paper]() Recorded evaluation configuration: {"Inference Budget (b)":"512","n":"200","Hardware":"A100","Mode":"pilot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.