MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH500
99.32Score
39
MATH
87Exact Match Accuracy
39
MATH
71Accuracy
39
MATH-500 1 (test)
93.2Accuracy
38
MATH 500
97.6Accuracy
38
MATH
84.5EM
38
MATH-500
15,508Mean Peak GPU Memory (MB)
37
MATH-500
93.5Accuracy
37
MATH500
98.2Mean Accuracy (MATH500)
36
MATH
6Attack Success Rate (ASR)
36
MATH 500
99.2Accuracy
36
MATH 500
97Accuracy
36
MATH Easy
97.8Accuracy
36
Minerva MATH500
65.2Score
36
Math (test)
80.9Accuracy
36
MATH 200 samples (test)
75Accuracy
36
MATH (test)
58.7Mistral-7B Accuracy
35
MATH 4-shot
69.6Score
35
MATH500
92.1Accuracy (MATH500)
34
Math Domain-level stream
47.91AA
34
MATH500
47Accuracy
34
MATH500
91.2Accuracy
34
MATH in-domain (test)
80.8Accuracy
34
MATH-500
92.6Accuracy (MATH-500)
33
MATH500
937.4Tokens
33