Document Summarization on SummScreenFD (test)
39.54ROUGE-1Top Down Transformer
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Top Down TransformerPooling strategy=AdaPool, Oracle usage=OracleAdaPool2022.03 | 39.54 | 10.08 | 33.59 | — | — | |
| Extractive OracleOracle usage=true2022.03 | 38.8 | 11.5 | 33.9 | — | — | |
| CachEDBackbone=BART-large, Parameters=406M2025.01 | 37.2 | 9.1 | 20.1 | 61.59 | — | |
| Top Down TransformerPooling strategy=AdaPool2022.03 | 36.84 | 9.19 | 31.12 | — | — | |
| CachEDBackbone=BART-base, Parameters=139M2025.01 | 36.6 | 8.8 | 19.9 | 61.3 | — | |
| Top Down TransformerPooling strategy=AvgPool2022.03 | 35.84 | 8.86 | 30.62 | — | — | |
| DIALOGLMFine-tuned parameters (millions)=3402024.04 | 35.75 | 8.27 | 30.76 | — | — | |
| SELECT & SUMMFine-tuned parameters (millions)=161, Classification mode=Zero-Shot2024.04 | 35.61 | 8.58 | 31.13 | — | — | |
| SLEDBackbone=BART-large, Parameters=406M2025.01 | 35.2 | 8.7 | 19.4 | 59.9 | — | |
| SLEDFine-tuned parameters (millions)=4062024.04 | 35.2 | 8.7 | 19.4 | — | — | |
| UnlimiformerBackbone=BART-base, Parameters=139M2025.01 | 34.7 | 8.5 | 19.9 | 58.5 | — | |
| SFTBackbone=BART-large, Parameters=406M2025.01 | 34.1 | 7.5 | 18.9 | 59.6 | — | |
| SFTBackbone=LED-base, Parameters=162M2025.01 | 33.8 | 8.3 | 19.8 | 59.5 | — | |
| UnlimiformerBackbone=PRIMERA, Parameters=447M2025.01 | 33.3 | 7.6 | 18.9 | 57.7 | — | |
| CachEDBackbone=T5-large, Parameters=770M2025.01 | 32.8 | 8.2 | 19.6 | 60.2 | — | |
| SLEDBackbone=BART-base, Parameters=139M2025.01 | 32.7 | 7.9 | 19.1 | 58.4 | — | |
| SUMMN2021.10 | 32.48 | 5.85 | 27.55 | — | — | |
| SUMMNFine-tuned parameters (millions)=4002024.04 | 32.48 | 5.85 | 27.55 | — | — | |
| SFTBackbone=BART-base, Parameters=139M2025.01 | 31.5 | 6.7 | 18.1 | 58.2 | — | |
| SFTBackbone=T5-large, Parameters=770M2025.01 | 28.9 | 5.3 | 16.9 | 58.1 | — | |
| Hybrid (BART + Oracle Content Selection)Backbone=BART, Selection=Oracle Content Selection, Oracle usage=true2022.03 | 26.4 | 5 | 23.3 | — | — | |
| Longformer2022.03 | 25.9 | 4.2 | 23.8 | — | — | |
| Longformer+ATT2021.10 | 25.9 | 4.2 | 23.8 | — | — | |
| Hybrid (BART + Content Selection)Backbone=BART, Selection=Content Selection2022.03 | 25.3 | 3.9 | 23.1 | — | — | |
| NN+BM25+Neural2021.10 | 25.3 | 3.9 | 23.1 | — | — | |
| CachEDbackbone=BART Large, size=406M, category=Long Context Modeling2025.05 | — | — | — | 61.59 | — | |
| Mistral Largemode=Zero-Shot, size=123B, category=Long Context Modeling2025.05 | — | — | — | 57.23 | — | |
| NEXUSSUMbackbone=Mistral Large, size=123B, category=Multi-LLM Agent2025.05 | — | — | — | 61.59 | — | |
| Select and Summbackbone=LED-Large, size=459M, category=Extractive-to-Abstractive2025.05 | — | — | — | 57.46 | — | |
| SLEDbackbone=BART Large, size=406M, category=Long Context Modeling2025.05 | — | — | — | 59.9 | — | |
| Unlimiformerbackbone=BART Base, size=139M, category=Long Context Modeling2025.05 | — | — | — | 58.5 | — |