2Wiki
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
2WIKI
86EM
260
2Wiki
74.9Exact Match
215
2Wiki (test)
61.8EM Accuracy
59
2Wiki
71.74Average Accuracy (AA)
56
2Wiki
43.6Accuracy (2Wiki)
44
2Wiki
62EM
42
2Wiki
96.13Recall@5
42
2Wiki (test)
69.7F1 Score
34
2Wiki
90.1Recall@2
32
2Wiki
78.8ASR
30
2Wiki (out-of-domain)
47.6Accuracy
28
2Wiki 100K context
78.91Accuracy
25
2Wiki (test)
45.1Accuracy
24
2Wiki
45.2Success Rate
22
2Wiki 18
43.6Exact Match (EM)
20
2Wiki
58.7Token-Level F1
20
2Wiki 30K context
73.7Accuracy
19
2Wiki 10K context
72.2Accuracy
19
2Wiki
58pass@1
18
2Wiki
46.9Exact Match (EM)
18
2wiki
61.3F1 Score
17
2Wiki
70.5Accuracy
17
2Wiki
59MBE
17
2Wiki
48.89EM
16
2wiki
81.21P@1
16