Email Subject Line Generation on AESLC (dev)
35.93ROUGE-1Human Annotation
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Human AnnotationReference=Two human annotations2019.06 | 35.93 | 17.76 | 33.55 | 21.74 | — | |
| ESQE2019.06 | 25.41 | 11.34 | 25.07 | 9.83 | — | |
| Our SystemReference=Two human annotations2019.06 | 25.39 | 10.94 | 24.72 | 13.04 | — | |
| Original SubjectReference=Two human annotations2019.06 | 24.38 | 10.15 | 23 | 16.49 | — | |
| Human Annotation2019.06 | 23.43 | 9.71 | 22.17 | 10.87 | — | |
| See et al.Reference=Two human annotations, Year=20172019.06 | 23.37 | 7.36 | 20.99 | 16.27 | — | |
| Hsu et al.Reference=Two human annotations, Year=20182019.06 | 22.98 | 7.07 | 19.95 | 18.83 | — | |
| LexRankReference=Two human annotations2019.06 | 21.82 | 10.83 | 19.78 | 20.82 | — | |
| LEAD-2Reference=Two human annotations2019.06 | 18.88 | 9.47 | 17.41 | 20.7 | — | |
| TextRankReference=Two human annotations2019.06 | 18.29 | 8.04 | 16.45 | 17 | — | |
| See et al. (2017)2019.06 | 18.02 | 5.73 | 16.63 | 10.83 | — | |
| Hsu et al. (2018)2019.06 | 16.59 | 4.67 | 15.12 | 13.22 | — | |
| Shang et al.Reference=Two human annotations, Year=20182019.06 | 16.28 | 6.14 | 15.07 | 12.12 | — | |
| Paulus et al.Reference=Two human annotations, Year=20182019.06 | 15.12 | 4.62 | 13.98 | 10.82 | — | |
| Paulus et al. (2018)2019.06 | 14.08 | 5.09 | 13.36 | 9.07 | — | |
| Narayan et al. (2018a)2019.06 | 13.52 | 3.27 | 13.33 | 4.64 | — | |
| LexRank2019.06 | 13.02 | 4.96 | 11.89 | 10.84 | — | |
| Narayan et al.Reference=Two human annotations, Year=2018a2019.06 | 11.33 | 1.45 | 11.14 | 4.9 | — | |
| LEAD-22019.06 | 11.28 | 4.61 | 10.48 | 10.76 | — | |
| TextRank2019.06 | 11.12 | 3.75 | 10.15 | 9.19 | — | |
| Shang et al. (2018)2019.06 | 10.56 | 3.28 | 9.92 | 6.17 | — | |
| Hsu et al. (2018)batch size=162019.06 | — | — | — | — | 1.44 | |
| Human Annotation2019.06 | — | — | — | — | 2.53 | |
| LEAD-22019.06 | — | — | — | — | 1.56 | |
| LexRank2019.06 | — | — | — | — | 1.57 | |
| Narayan et al. (2018a)2019.06 | — | — | — | — | 1.53 | |
| Original Subject2019.06 | — | — | — | — | 2.52 | |
| Our System2019.06 | — | — | — | — | 2.4 | |
| Paulus et al. (2018)implementation=Keneshloo et al. (2018)2019.06 | — | — | — | — | 2.3 | |
| See et al. (2017)batch size=162019.06 | — | — | — | — | 2.22 | |
| Shang et al. (2018)2019.06 | — | — | — | — | 2.1 | |
| TextRank2019.06 | — | — | — | — | 1.59 |