Summarization on GovReport SCROLLS (test)
48.18ROUGE-1BART
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BARTBackbone=BART-base, Evaluation Protocol=FT (standard), Input Length (n)=1024, Attention=Dense2026.05 | 48.18 | 23.55 | 29.26 | 31.64 | |
| Block-ChaCALBackbone=BART-base, Evaluation Protocol=FT (modified), Input Length (n)=1024, Attention=Blockwise substitution in decoder's first layer, m=n//32026.05 | 38.46 | 11.52 | 20.12 | 20.43 | |
| BARTBackbone=BART-base, Evaluation Protocol=zero-shot, Input Length (n)=1024, Attention=Dense2026.05 | 29.68 | 10.84 | 16.32 | 16.99 |