# Reasoning on GSM8K, MATH-500, AIME24, AIME25, GPQA-D Source: [Reasoning on GSM8K, MATH-500, AIME24, AIME25, GPQA-D]() Collection index: [llms.txt]() Data updated: 2026-03-18T09:01:43.148Z Aggregate evaluation of model performance across a suite of reasoning benchmarks: GSM8K, MATH-500, AIME24, AIME25, and GPQA-D. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Reasoning | | Dataset (including any stated split) | Reasoning Benchmarks GSM8K, MATH-500, AIME24, AIME25, GPQA-D | | Metric | GSM8K Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [GSM8K Accuracy](): Higher is better; unit: PERCENT. - [MATH-500 Accuracy](): Higher is better; unit: PERCENT. - [AIME24 Accuracy](): Higher is better; unit: PERCENT. - [AIME25 Accuracy](): Higher is better; unit: PERCENT. - [GPQA-D Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. - [Recovery Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | GSM8K Accuracy | Result date | Report paper | | --- | --- | --- | --- | | BF16 | 95.15 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 94.92 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 94.84 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 94.77 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 94.77 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 94.69 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 94.69 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 94.69 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 94.62 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 94.62 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 94.54 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 94.47 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 94.47 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 94.47 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 94.39 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 94.39 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 94.24 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 94.09 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 94.09 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 94.09 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 94.01 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 93.78 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 93.71 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 93.71 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 93.56 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 93.4 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 93.4 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 93.4 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 93.25 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 93.1 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 93.03 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 92.27 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 92.12 | Not recorded | 69ba31900636e5bf4afc0884 | ### BF16 GSM8K Accuracy: 95.15. Result ID: 69ba69d847f34ce924a4816f. [Reporting paper]() Recorded evaluation configuration: {"Bits":"BF16"}. This record does not assert independent reproduction. ### SmoothQuant GSM8K Accuracy: 94.92. Result ID: 69ba69d847f34ce924a48175. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### BATQuant GSM8K Accuracy: 94.84. Result ID: 69ba69d847f34ce924a48177. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### BATQuant GSM8K Accuracy: 94.77. Result ID: 69ba69d847f34ce924a4817f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### BATQuant GSM8K Accuracy: 94.77. Result ID: 69ba69d847f34ce924a4818f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### SpinQuant GSM8K Accuracy: 94.69. Result ID: 69ba69d847f34ce924a48172. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### SmoothQuant GSM8K Accuracy: 94.69. Result ID: 69ba69d847f34ce924a4817d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### BRQ GSM8K Accuracy: 94.69. Result ID: 69ba69d847f34ce924a48183. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### FlatQuant GSM8K Accuracy: 94.62. Result ID: 69ba69d847f34ce924a48174. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### BATQuant GSM8K Accuracy: 94.62. Result ID: 69ba69d847f34ce924a48187. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### FlatQuant GSM8K Accuracy: 94.54. Result ID: 69ba69d847f34ce924a48184. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### QuaRot GSM8K Accuracy: 94.47. Result ID: 69ba69d847f34ce924a48171. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### SpinQuant GSM8K Accuracy: 94.47. Result ID: 69ba69d847f34ce924a48182. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### GPTQ GSM8K Accuracy: 94.47. Result ID: 69ba69d847f34ce924a48186. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### GPTQ GSM8K Accuracy: 94.39. Result ID: 69ba69d847f34ce924a48176. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### SmoothQuant GSM8K Accuracy: 94.39. Result ID: 69ba69d847f34ce924a48185. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### GPTQ GSM8K Accuracy: 94.24. Result ID: 69ba69d847f34ce924a4817e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### QuaRot GSM8K Accuracy: 94.09. Result ID: 69ba69d847f34ce924a48179. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### QuaRot GSM8K Accuracy: 94.09. Result ID: 69ba69d847f34ce924a48181. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### FlatQuant GSM8K Accuracy: 94.09. Result ID: 69ba69d847f34ce924a4818c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### QuaRot GSM8K Accuracy: 94.01. Result ID: 69ba69d847f34ce924a48189. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### RTN GSM8K Accuracy: 93.78. Result ID: 69ba69d847f34ce924a48180. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### RTN GSM8K Accuracy: 93.71. Result ID: 69ba69d847f34ce924a48170. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### BRQ GSM8K Accuracy: 93.71. Result ID: 69ba69d847f34ce924a48173. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### BRQ GSM8K Accuracy: 93.56. Result ID: 69ba69d847f34ce924a4818b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### SpinQuant GSM8K Accuracy: 93.4. Result ID: 69ba69d847f34ce924a4817a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### FlatQuant GSM8K Accuracy: 93.4. Result ID: 69ba69d847f34ce924a4817c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### GPTQ GSM8K Accuracy: 93.4. Result ID: 69ba69d847f34ce924a4818e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### SpinQuant GSM8K Accuracy: 93.25. Result ID: 69ba69d847f34ce924a4818a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### RTN GSM8K Accuracy: 93.1. Result ID: 69ba69d847f34ce924a48178. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### SmoothQuant GSM8K Accuracy: 93.03. Result ID: 69ba69d847f34ce924a4818d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### BRQ GSM8K Accuracy: 92.27. Result ID: 69ba69d847f34ce924a4817b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### RTN GSM8K Accuracy: 92.12. Result ID: 69ba69d847f34ce924a48188. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.