# Abstractive Question Summarization on MEQSUM 1.0 (test) Source: [Abstractive Question Summarization on MEQSUM 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:00:33.424Z Evaluation of abstractive question summarization performance on the test split of the MEQSUM 1.0 dataset, measured using ROUGE metrics (ROUGE-1, ROUGE-2, ROUGE-L). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Abstractive Question Summarization | | Dataset (including any stated split) | MEQSUM 1.0 (test) | | Metric | ROUGE-1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ROUGE-1](): Higher is better; unit: PERCENT. - [ROUGE-2](): Higher is better; unit: PERCENT. - [ROUGE-L](): Higher is better; unit: PERCENT. ## Recorded results | Method | ROUGE-1 | Result date | Report paper | | --- | --- | --- | --- | | ProphetNet + QTR + QFR | 45.52 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + QFR | 45.36 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + Q-type + Q-focus | 44.67 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + Q-focus | 44.62 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + QTR | 44.6 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + Q-type | 44.4 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + ROUGE-L | 44.33 | Not recorded | 68fa6a471e209131cea274bb | | SOTA (Ben Abacha and Demner-Fushman, 2019) | 44.16 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet | 43.87 | Not recorded | 68fa6a471e209131cea274bb | | MINILM | 43.13 | Not recorded | 68fa6a471e209131cea274bb | | BART-LARGE | 42.3 | Not recorded | 68fa6a471e209131cea274bb | | SOTA\* | 40 | Not recorded | 68fa6a471e209131cea274bb | | PEGASUS | 39.06 | Not recorded | 68fa6a471e209131cea274bb | | T5-BASE | 38.92 | Not recorded | 68fa6a471e209131cea274bb | | Pointer Generator (PG) | 32.41 | Not recorded | 68fa6a471e209131cea274bb | | Seq2Seq + Attention | 28.11 | Not recorded | 68fa6a471e209131cea274bb | | BertSumm | 26.24 | Not recorded | 68fa6a471e209131cea274bb | | Transformer | 25.84 | Not recorded | 68fa6a471e209131cea274bb | | Seq2Seq | 25.28 | Not recorded | 68fa6a471e209131cea274bb | ### ProphetNet + QTR + QFR ROUGE-1: 45.52. Result ID: 698e282affc3901de09b3f24. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Proposed Approach","reward":"QTR + QFR"}. This record does not assert independent reproduction. ### ProphetNet + QFR ROUGE-1: 45.36. Result ID: 698e282affc3901de09b3f23. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Proposed Approach","reward":"Question-focus recognition"}. This record does not assert independent reproduction. ### ProphetNet + Q-type + Q-focus ROUGE-1: 44.67. Result ID: 698e282affc3901de09b3f21. [Reporting paper]() Recorded evaluation configuration: {"category":"Joint Learning"}. This record does not assert independent reproduction. ### ProphetNet + Q-focus ROUGE-1: 44.62. Result ID: 698e282affc3901de09b3f20. [Reporting paper]() Recorded evaluation configuration: {"category":"Joint Learning"}. This record does not assert independent reproduction. ### ProphetNet + QTR ROUGE-1: 44.6. Result ID: 698e282affc3901de09b3f22. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Proposed Approach","reward":"Question-type identification"}. This record does not assert independent reproduction. ### ProphetNet + Q-type ROUGE-1: 44.4. Result ID: 698e282affc3901de09b3f1f. [Reporting paper]() Recorded evaluation configuration: {"category":"Joint Learning"}. This record does not assert independent reproduction. ### ProphetNet + ROUGE-L ROUGE-1: 44.33. Result ID: 698e282affc3901de09b3f1e. [Reporting paper]() Recorded evaluation configuration: {"category":"Baseline","RL\_reward":"ROUGE-L"}. This record does not assert independent reproduction. ### SOTA (Ben Abacha and Demner-Fushman, 2019) ROUGE-1: 44.16. Result ID: 698e282affc3901de09b3f15. [Reporting paper]() Recorded evaluation configuration: {"category":"Baseline","implementation":"original"}. This record does not assert independent reproduction. ### ProphetNet ROUGE-1: 43.87. Result ID: 698e282affc3901de09b3f1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### MINILM ROUGE-1: 43.13. Result ID: 698e282affc3901de09b3f1c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### BART-LARGE ROUGE-1: 42.3. Result ID: 698e282affc3901de09b3f1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### SOTA\* ROUGE-1: 40. Result ID: 698e282affc3901de09b3f16. [Reporting paper]() Recorded evaluation configuration: {"category":"Baseline","implementation":"trained on same data"}. This record does not assert independent reproduction. ### PEGASUS ROUGE-1: 39.06. Result ID: 698e282affc3901de09b3f1a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### T5-BASE ROUGE-1: 38.92. Result ID: 698e282affc3901de09b3f19. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Pointer Generator (PG) ROUGE-1: 32.41. Result ID: 698e282affc3901de09b3f14. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Seq2Seq + Attention ROUGE-1: 28.11. Result ID: 698e282affc3901de09b3f13. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### BertSumm ROUGE-1: 26.24. Result ID: 698e282affc3901de09b3f18. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Transformer ROUGE-1: 25.84. Result ID: 698e282affc3901de09b3f17. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Seq2Seq ROUGE-1: 25.28. Result ID: 698e282affc3901de09b3f12. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.