SVAMP
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SVAMP
59.8Accuracy
9
SVAMP
96.7Accuracy (Acc)
8
SVAMP
53.8Answer Correctness Rate
8
SVAMP
68.8ROC-AUC (128)
8
SVAMP
78.33Kazakh (Cyrl) Accuracy
7
SVAMP 10 low-resourced languages FLORES-200
7.33Kazakh (Cyrl) Score
7
SVAMP 10 low-resourced languages FLORES-200 (test)
44Kazakh (Cyrillic) Accuracy
7
SVAMP
66.12Exact Match Accuracy
7
SVAMP
95.6Accuracy
7
SVAMP
55.9Exact-match Accuracy
6
SVAMP
96.62Pass@10
6
SVAMP
0.973Pass@20
6
SVAMP
65.83AUROC Score
6
SVAMP (test)
4.63GPT-4 Score
6
SVAMP hard (test)
48.2Accuracy
6
SVAMP
93.64Accuracy (Source Lib.)
5
SVAMP original GSM8k-Aug rationale format
64.9Pass@1
5
SVAMP
60.78Pass@1
5
SVAMP
86.6Accuracy (SVAMP)
5
SVAMP
92.4Accuracy
5
SVAMP
90.3Accuracy
4
SVAMP
100Accuracy under Attack
4
SVAMP
80.5SVAMP Accuracy (Ref)
4
SVAMP (test)
57.4Exact Match (EM)
4
SVAMP
95Accuracy
4