IOI
Benchmarks
TaskDatasetSOTA resultTrendResults
IOI
0.15Last-Token KL Divergence
40
IOI
0Dissimilarity (dis.)
32
IOI
3.54CPR
30
IOI
3.491CPR
16
IOI
91CPR Faithfulness Score
14
IOI
83.6AUC
12
IOI 2 v1 (Eva)
100Accuracy
6
IOI v1 (Infer)
1Accuracy
6
IOI
0.668KL Div
6
IOI 2025
585Score
5
IOI 2025
100Score S
4
IOI
76Accuracy
4
IOI Pythia-160M
0.07Mean Silhouette Score
3
IOI
0.03Silhouette Score (Mean)
3
IOI
96.74Sparsity
3
IOI 400 examples v1
0.22KL Divergence
3
IOI 200 examples v1
0.25KL Divergence
3
IOI 2025
439.28Score
2
IOI 100K examples v1
0.2KL Divergence
2
IOI evaluation episodes (held-out)
2.976Policy Score
1