Xsum
Benchmarks
TaskDatasetSOTA resultTrendResults
XSUM (test)
30.4Throughput (tokens/s)
8
XSum Faith
38.8Spearman Correlation
8
F-Xsum
24.5Kendall's Tau
8
XSumFaith
26.6Kendall's Tau
8
XSUM
11.4NOUN
8
XSum DPO-optimized adversarial (test)
51.56Accuracy
7
XSum
0.761MiniCheck Score
7
XSUM (5000 random instances)
1.73Speedup Factor
7
Xsum BART-base (dev)
40.61ROUGE-1
7
XSUM
0.268PCC
7
XSum
98ASR
6
XSUM
0Bias Flip Rate
6
XSum
27.27Throughput (sample/s)
6
XSum 18.5:1
6.69Cohen's d
6
XSum
8.79ROUGE-2
6
XSum
19Accuracy
6
XSum
21.6ROUGE-L
6
XSum
33.1TPR @ 0.1% FPR
6
XSum (test)
83.44F1 (Expand)
5
XSum 21w (test)
32.4ROUGE-1
5
XSum
40.2ROUGE-1
5
XSum
20.4ROUGE-L
5
XSum
93.37TPR
5
XSum
5.38GFLOPs/token
5
XSum Fast-DetectGPT benchmark (300 samples)
0.958AUROC
5