TL;DR
Benchmarks
TaskDatasetSOTA resultTrendResults
TL;DR
91.8Winrate
59
TL;DR (test)
82.5Win Rate
49
TL;DR (test)
68.8Win Rate
36
TL;DR (test)
0.427RK
16
TL;DR (distillation set)
27.24Word Count
16
TL;DR
94.2AlignScore
15
TL;DR (test)
87.2Win Rate (0% Flip)
14
TL;DR Seen (n=100)
62.3Accuracy
14
TL;DR
82.6Coverage
12
TL;DR
0.421Rank Correlation (RK)
11
TL;DR
43Completeness
11
TL;DR
83.4Win Rate (0% Flip Rate)
10
TL;DR Overall n=150
62.9Accuracy
7
TL;DR Unseen (n=150)
62.4Accuracy
7
TL;DR n=150 Seen
63.3Accuracy
7
TL;DR n=100 Unseen
61.5Accuracy
7
TL;DR
92.8Win Rate
6
TL;DR
0.46Kendall's Tau
5
TL;DR
0.44Kendall's Tau
5
TL;DR (test)
68.56Win Rate (GPT-4o)
4
TL;DR
64.4GRA (%)
4
TL;DR
50.5Winrate
4
TL;DR
3False Alarm Rate
3
TL;DR (val)
—Primary metric
0
TL;DR (test)
—Primary metric
0