WeMath
Benchmarks
TaskDatasetSOTA resultTrendResults
WeMath
84Accuracy
317
WeMath
98.7Accuracy
228
WeMath
78Accuracy
199
WeMath
98.7Accuracy
193
WeMath
71.3Accuracy
34
WeMath
85.11Pass@1
30
WeMath 525 samples
78.5Accuracy
24
WeMath strict
44.8Accuracy
18
WeMath mini (test)
79.5Accuracy
18
WeMath
63.9Macro F1
18
WeMath (test)
72.15Accuracy
17
WeMath
82.6Accuracy
16
WeMath strict
39Score
12
WeMath Loose
79Score
10
WeMath
68.29Accuracy
8
WeMath
36.33WeMath-S Score
8
WeMath
71.77Accuracy
8
WeMath (test)
82.9Accuracy
6
WeMath-L
82.19Score
6
WeMath-S
68.86Score
6
WeMath
61.6Accuracy
6
WeMath loose
52.1Accuracy
6
WeMath
24.9FISI F1 Score
6
WeMath v1.0 (test)
31.8Accuracy
5
WeMath 19
61.52Macro Average Score
2