Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Summarization research benchmarks – Page 14 · SOTA2 Research

Summarization

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

XLSUM Hindi (hi) (test)

29.83ROUGE-1

5

Jun 9, 2026

XLSUM Thai (th) (test)

Llama3 8B-I + Adaptive LATB

30.97ROUGE-1

5

Jun 9, 2026

XLSUM French (fr) (test)

Llama3 8B-I + LATB

20.13ROUGE-1

5

Jun 9, 2026

XLSUM Chinese (zh) (test)

Llama3 8B-I + LATB

0.207ROUGE-1

5

Jun 9, 2026

XLSUM Russian (ru) (test)

Llama3 8B-I + Adaptive LATB

21ROUGE-1

5

Jun 9, 2026

arXiv 163w (test)

Llama-3.1-8B AR

43.5ROUGE-1

5

Jun 2, 2026

PubMed 205w (test)

Llama-3.1-8B AR

45.1ROUGE-1

5

Jun 2, 2026

CNN/DM 55w (test)

Llama-3.1-8B AR

37.8ROUGE-1

5

Jun 2, 2026

XSum 21w (test)

32.4ROUGE-1

5

Jun 2, 2026

4.87Factuality

5

May 29, 2026

13.9ROUGE-L

5

May 28, 2026

19.3ROUGE-L

5

May 27, 2026

20.4ROUGE-L

5

May 27, 2026

LongBench Summarization

18.08Accuracy

5

May 19, 2026

0.022ECE (Instance)

5

Apr 21, 2026

OmniScore Evaluation Set

Gemini-3.1-Flash-Lite

0.91MAE

5

Apr 8, 2026

41.31Score

5

Mar 30, 2026

Average arXiv, SciGen, WikiHow

Extract-to-Abstract (E2A)

4.42Conciseness

5

Feb 26, 2026

Extract-to-Abstract (E2A)

4.45Conciseness

5

Feb 26, 2026

Self-Consistency (SC)

4.3Conciseness

5

Feb 26, 2026

Extract-to-Abstract (E2A)

4.5Conciseness

5

Feb 26, 2026

Human Evaluation

4.4Coherence

5

Feb 26, 2026

0.46Kendall's Tau

5

Feb 26, 2026

XWikis en-en (test)

31.28RL Score

5

Feb 26, 2026

Newsroom Non-Biased

22ROUGE-L

5

Feb 26, 2026

Page 14 of 19

Popular datasets

CNN/DM Xsum SamSum XLSum Pubmed

Follow for updates

Join SOTA2 Contact