Summary Generation on AmaSum 4 domains
89.4QA(References)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| (References)2023.05 | 89.4 | — | — | 86.68 | 65.59 | |
| (Oracle)2023.05 | 14.04 | 14.36 | 26.04 | 26.38 | 76.35 | |
| HERCULESabsSystem Category=Abstractive2023.05 | 7.67 | 2.05 | 11.77 | 25.23 | 82.72 | |
| HERCULESextSystem Category=Extractive2023.05 | 6.94 | 3.04 | 12.51 | 24.38 | 84.05 | |
| InstructGPTSystem Category=Abstractive2023.05 | 6.89 | 2.71 | 13.64 | 21.87 | 45.63 | |
| COOPSystem Category=Abstractive2023.05 | 6.14 | 2.79 | 14.12 | 22.51 | 58.35 | |
| BiMeanVAESystem Category=Abstractive2023.05 | 5.66 | 2.04 | 12.49 | 21.78 | 52.54 | |
| LexRankSystem Category=Extractive2023.05 | 4.95 | 2.66 | 12.2 | 23.49 | 67.2 | |
| CopyCatSystem Category=Abstractive2023.05 | 4.39 | 1.5 | 11.21 | 22.99 | 63.01 | |
| CentroidSystem Category=Extractive2023.05 | 3.89 | 2 | 11.21 | 23.47 | 64.63 | |
| QTSystem Category=Extractive2023.05 | 3.62 | 1.51 | 11.41 | 22.42 | 66.21 | |
| RandomSystem Category=Extractive2023.05 | 3.09 | 1.02 | 9.46 | 22.41 | 59.17 | |
| SemAESystem Category=Extractive2023.05 | 2.66 | 1.58 | 11.26 | 21.83 | 57.19 |