Abstractive Question Summarization on MEQSUM 1.0 (test)
45.52ROUGE-1ProphetNet + QTR + QFR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ProphetNet + QTR + QFRcategory=Proposed Approach, reward=QTR + QFR2021.07 | 45.52 | 27.54 | 48.19 | |
| ProphetNet + QFRcategory=Proposed Approach, reward=Question-focus recognition2021.07 | 45.36 | 27.33 | 47.96 | |
| ProphetNet + Q-type + Q-focuscategory=Joint Learning2021.07 | 44.67 | 26.72 | 47.34 | |
| ProphetNet + Q-focuscategory=Joint Learning2021.07 | 44.62 | 26.61 | 47.28 | |
| ProphetNet + QTRcategory=Proposed Approach, reward=Question-type identification2021.07 | 44.6 | 26.69 | 47.38 | |
| ProphetNet + Q-typecategory=Joint Learning2021.07 | 44.4 | 26.63 | 47.05 | |
| ProphetNet + ROUGE-Lcategory=Baseline, RL_reward=ROUGE-L2021.07 | 44.33 | 26.32 | 46.9 | |
| SOTA (Ben Abacha and Demner-Fushman, 2019)category=Baseline, implementation=original2021.07 | 44.16 | 27.64 | 42.78 | |
| ProphetNetcategory=Baseline2021.07 | 43.87 | 25.99 | 46.52 | |
| MINILMcategory=Baseline2021.07 | 43.13 | 26.03 | 46.39 | |
| BART-LARGEcategory=Baseline2021.07 | 42.3 | 24.83 | 43.74 | |
| SOTA*category=Baseline, implementation=trained on same data2021.07 | 40 | 24.13 | 38.56 | |
| PEGASUScategory=Baseline2021.07 | 39.06 | 20.18 | 42.05 | |
| T5-BASEcategory=Baseline2021.07 | 38.92 | 21.29 | 40.56 | |
| Pointer Generator (PG)category=Baseline2021.07 | 32.41 | 19.37 | 36.53 | |
| Seq2Seq + Attentioncategory=Baseline2021.07 | 28.11 | 17.24 | 27.82 | |
| BertSummcategory=Baseline2021.07 | 26.24 | 16.2 | 30.59 | |
| Transformercategory=Baseline2021.07 | 25.84 | 13.66 | 29.12 | |
| Seq2Seqcategory=Baseline2021.07 | 25.28 | 14.39 | 24.64 |