MAB
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MAB code
69.6TS
18
MAB FC-SH (262K context) v3 (full)
93Accuracy (SubEM)
9
MAB FC-SH 262K v3 (test)
93Accuracy
8
MAB FC-MH 262K context v3 (full)
27Accuracy (SubEM)
7
MAB FC-MH, 262K v3 (test)
27Accuracy
5
MAB 1-bounded
1Competitive Ratio
2
MAB Multi-task
1Non-adaptive Upper Bound
1
MAB with paid observations
—Primary metric
0