Extreme Summarization on SciTLDR 1.0 (test)
52.4ROUGE-1Oracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleInput source=AIC, ROUGE protocol=max2020.04 | 52.4 | 29 | 42.9 | |
| OracleInput source=Abstract-only, ROUGE protocol=max2020.04 | 47.7 | 24.7 | 38.5 | |
| CATTSInput source=AIC, ROUGE protocol=max2020.04 | 44.9 | 22.6 | 37.3 | |
| CATTSXSUMInput source=AIC, ROUGE protocol=max2020.04 | 44.6 | 21.7 | 36.5 | |
| CATTSXSUMInput source=Abstract-only, ROUGE protocol=max2020.04 | 44.3 | 21.3 | 35.9 | |
| CATTSInput source=Abstract-only, ROUGE protocol=max2020.04 | 43.8 | 20.9 | 35.5 | |
| BARTXSUMInput source=AIC, ROUGE protocol=max2020.04 | 43.7 | 21.4 | 36 | |
| BARTInput source=Abstract-only, ROUGE protocol=max2020.04 | 43.3 | 20.8 | 35 | |
| BARTInput source=AIC, ROUGE protocol=max2020.04 | 42.9 | 20.8 | 35.1 | |
| MatchSumInput source=Abstract-only, ROUGE protocol=max2020.04 | 42.7 | 20 | 34 | |
| BARTXSUMInput source=Abstract-only, ROUGE protocol=max2020.04 | 42.5 | 21.1 | 34.9 | |
| MatchSumInput source=AIC, ROUGE protocol=max2020.04 | 38.6 | 16.4 | 30.1 | |
| BERTSUMEXTInput source=Abstract-only, ROUGE protocol=max2020.04 | 38.5 | 16.6 | 30.5 | |
| BERTSUMEXTInput source=AIC, ROUGE protocol=max2020.04 | 36.2 | 14.7 | 28.5 | |
| CATTSXSUMInitialization=XSUM2020.04 | 31.7 | 11.1 | 25 | |
| CATTSInitialization=Default2020.04 | 31.5 | 11 | 24.9 | |
| BARTInitialization=Default2020.04 | 31.1 | 10.7 | 24.4 | |
| BARTXSUMInitialization=XSUM2020.04 | 30.1 | 10.7 | 24.1 | |
| PACSUMInput source=AIC, ROUGE protocol=max2020.04 | 28.7 | 9.8 | 21.9 | |
| PACSUMInput source=Abstract-only, ROUGE protocol=max2020.04 | 19.3 | 4 | 15.1 |