Document Summarization on GovReport
60.64ROUGE-1BigBird
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BigBird2026.02 | 60.64 | 24.81 | 50.01 | — | |
| BART HEPOS (Sinkhorn)Maximum input length=102402021.10 | 56.86 | 22.62 | 53.82 | — | |
| SUMMN2021.10 | 56.77 | 23.25 | 53.9 | — | |
| GloSA-sum2026.02 | 55.5 | 26 | 51 | — | |
| BART (Sinkhorn)Maximum input length=51202021.10 | 55.45 | 21.45 | 52.48 | — | |
| BART HEPOS (LSH)Maximum input length=71682021.10 | 55 | 21.13 | 51.67 | — | |
| BART (LSH)Maximum input length=40962021.10 | 54.75 | 21.36 | 51.27 | — | |
| BART (Stride)Maximum input length=40962021.10 | 54.29 | 20.8 | 51.35 | — | |
| PEGASUS2026.02 | 54.29 | 20.8 | 51.35 | — | |
| TextRank2026.02 | 53.19 | 23.12 | 49.86 | — | |
| BART (Full)Maximum input length=10242021.10 | 52.83 | 20.5 | 50.14 | — | |
| BART2026.02 | 52.24 | 22.09 | 49.99 | — | |
| Lead-32026.02 | 50.94 | 19.53 | 48.45 | — | |
| MemSum2026.02 | 49.14 | 22.92 | 44.33 | — | |
| BART (LIN.)Maximum input length=30722021.10 | 44.84 | 13.87 | 41.94 | — | |
| CoABackbone=text-bison, Context Window=8k2024.06 | — | — | — | 26.11 | |
| CoABackbone=text-unicorn, Context Window=8k2024.06 | — | — | — | 26.48 | |
| CoABackbone=gemini-ultra, Context Window=8k2024.06 | — | — | — | 26.98 | |
| RAGBackbone=text-bison, Context Window=8k2024.06 | — | — | — | 20.83 | |
| RAGBackbone=text-unicorn, Context Window=8k2024.06 | — | — | — | 21.43 | |
| RAGBackbone=gemini-ultra, Context Window=8k2024.06 | — | — | — | 25.87 | |
| VanillaBackbone=text-bison, Context Window=8k2024.06 | — | — | — | 20.6 | |
| VanillaBackbone=text-unicorn, Context Window=8k2024.06 | — | — | — | 23.5 | |
| VanillaBackbone=gemini-ultra, Context Window=8k2024.06 | — | — | — | 26.18 | |
| VanillaBackbone=gemini-ultra, Context Window=32k2024.06 | — | — | — | 26.96 |