Xsum
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
XSUM randomly sampled
3.8U-PPL
20
XSum
1.84Speedup vs AR
19
XSum
49.02ROUGE-1
18
XSum
0.161Average Soft Constrained Reward (b=3)
18
XSum
16.57ROUGE-L
18
XSum
100TPR (FPR=0.01)
18
XSum Claude3 Opus
97.3TPR @ FPR 1%
18
XSum GPT4 Turbo
99.3TPR @ FPR 1%
18
XSum GPT4
79.3TPR @ FPR 1%
18
XSum GPT3.5 Turbo
96.7TPR @ FPR 1%
18
XSum
22.71R-L
17
XSum
91.7Accuracy
16
XSum 21w 1,000 samples (test)
32.4ROUGE-1 F1
16
XSum
32.4ROUGE-1 F1
16
XSum (val)
0.162ROUGE-2
16
XSUM in-domain (test)
20D3 Score
16
XSum GPT-J-6B
88.35AUROC
15
XSum
27.1ROUGE-2
14
XSum
22.47ROUGE-Lsum
14
XSum
17.9XSum Score
14
XSum In-Domain (IND-MGT)
91.7Accuracy
12
XSUM
88.7AlignScore
12
XSum
34.6ROUGE-1
12
XSum
0.73AUROC
12
XSum
19.67ROUGE-1
12