Abstractive Summarization on SciTLDR AIC
44.9ROUGE-1CATTS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CATTSFine-tuned=true, BLooP=false2026.03 | 44.9 | 22.6 | 37.3 | — | — | |
| BARTFine-tuned=true, BLooP=false2026.03 | 42.9 | 20.8 | 35.1 | — | — | |
| Llama 3.1Zero-shot=true, BLooP=false2026.03 | 36.64 | 11.53 | 32.09 | -4.1 | 1.93 | |
| Gemma 2Zero-shot=true, BLooP=false2026.03 | 36.55 | 11.31 | 32.2 | -4.04 | 1.99 | |
| Gemma 2 + BLooPZero-shot=true, BLooP=true2026.03 | 36.27 | 11.65 | 32.1 | -4.03 | 2.02 | |
| Llama 3.1 + BLooPZero-shot=true, BLooP=true2026.03 | 36.21 | 11.55 | 31.82 | -4.07 | 1.99 | |
| MistralZero-shot=true, BLooP=false2026.03 | 35.96 | 10.7 | 31.57 | -4.04 | 1.99 | |
| Mistral + BLooPZero-shot=true, BLooP=true2026.03 | 35.88 | 11.31 | 31.54 | -4.01 | 2.06 |