SummEval
Benchmarks
TaskDatasetSOTA resultTrendResults
SummEval
0.902Avg Spearman Rho
55
SummEval
57Coherence
41
SummEval
0.546Pearson Correlation
40
SummEval
46.6Spearman Correlation
36
SummEval (test)
66.3Pearson CC
22
SummEval 1.0 (test)
0.594Coherence (Spearman rho)
21
SummEval
68.9Coherence Accuracy
18
SummEval (Local)
0.687Coherence
16
SummEval Global
85.2Coherence
16
SummEval segment-level
60Coherence
15
SummEval
77.3Balanced Accuracy
15
SUMMEVAL-OP 1.0 (Round-II)
5FL (Fluency)
13
SummEval
88Top-k Recall
12
SummEval
0.72Completeness
11
SummEval (test)
0.668Coherence (Pearson r)
11
SummEval Rel
0.48Spearman's Rho
10
SummEval
0.676Spearman's Rho
10
SummEval (test)
0.575Coherence (Spearman ρ)
10
SummEval
0.448Spearman Correlation (COH)
10
SummEval
0.755AUROC
8
SummEval
55.9Accuracy
8
SummEval
0.495MSE
8
SummEval
66.7Pearson CC
8
SummEval
38.4Kendall's Tau
8
SummEval Relevance Domain
0.96Corr.
8