# Abstractive Question Summarization on MATINF English translated subset (test) Source: [Abstractive Question Summarization on MATINF English translated subset (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:00:33.444Z Evaluation of abstractive question summarization performance on the test split of the MATINF English translated subset, measured using ROUGE metrics (ROUGE-1, ROUGE-2, ROUGE-L). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Abstractive Question Summarization | | Dataset (including any stated split) | MATINF English translated subset (test) | | Metric | ROUGE-1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ROUGE-1](): Higher is better; unit: PERCENT. - [ROUGE-2](): Higher is better; unit: PERCENT. - [ROUGE-L](): Higher is better; unit: PERCENT. ## Recorded results | Method | ROUGE-1 | Result date | Report paper | | --- | --- | --- | --- | | ProphetNet + ROUGE-L | 48.17 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + QTR + QFR | 47.73 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + QFR | 47.53 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + QTR | 47.51 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + Q-type | 47.19 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + Q-type + Q-focus | 47.18 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet + Q-focus | 47.14 | Not recorded | 68fa6a471e209131cea274bb | | ProphetNet | 46.94 | Not recorded | 68fa6a471e209131cea274bb | | BART-LARGE | 42.52 | Not recorded | 68fa6a471e209131cea274bb | | PEGASUS | 40.05 | Not recorded | 68fa6a471e209131cea274bb | | T5-BASE | 39.66 | Not recorded | 68fa6a471e209131cea274bb | | MINILM | 35.6 | Not recorded | 68fa6a471e209131cea274bb | | BertSumm | 31.16 | Not recorded | 68fa6a471e209131cea274bb | | SOTA\* | 24.58 | Not recorded | 68fa6a471e209131cea274bb | | Pointer Generator (PG) | 23.31 | Not recorded | 68fa6a471e209131cea274bb | | Transformer | 22.25 | Not recorded | 68fa6a471e209131cea274bb | | Seq2Seq + Attention | 19.45 | Not recorded | 68fa6a471e209131cea274bb | | Seq2Seq | 17.77 | Not recorded | 68fa6a471e209131cea274bb | ### ProphetNet + ROUGE-L ROUGE-1: 48.17. Result ID: 698e282affc3901de09b3f30. [Reporting paper]() Recorded evaluation configuration: {"category":"Baseline","RL\_reward":"ROUGE-L"}. This record does not assert independent reproduction. ### ProphetNet + QTR + QFR ROUGE-1: 47.73. Result ID: 698e282affc3901de09b3f36. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Proposed Approach","reward":"QTR + QFR"}. This record does not assert independent reproduction. ### ProphetNet + QFR ROUGE-1: 47.53. Result ID: 698e282affc3901de09b3f35. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Proposed Approach","reward":"Question-focus recognition"}. This record does not assert independent reproduction. ### ProphetNet + QTR ROUGE-1: 47.51. Result ID: 698e282affc3901de09b3f34. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Proposed Approach","reward":"Question-type identification"}. This record does not assert independent reproduction. ### ProphetNet + Q-type ROUGE-1: 47.19. Result ID: 698e282affc3901de09b3f31. [Reporting paper]() Recorded evaluation configuration: {"category":"Joint Learning"}. This record does not assert independent reproduction. ### ProphetNet + Q-type + Q-focus ROUGE-1: 47.18. Result ID: 698e282affc3901de09b3f33. [Reporting paper]() Recorded evaluation configuration: {"category":"Joint Learning"}. This record does not assert independent reproduction. ### ProphetNet + Q-focus ROUGE-1: 47.14. Result ID: 698e282affc3901de09b3f32. [Reporting paper]() Recorded evaluation configuration: {"category":"Joint Learning"}. This record does not assert independent reproduction. ### ProphetNet ROUGE-1: 46.94. Result ID: 698e282affc3901de09b3f2f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### BART-LARGE ROUGE-1: 42.52. Result ID: 698e282affc3901de09b3f2d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### PEGASUS ROUGE-1: 40.05. Result ID: 698e282affc3901de09b3f2c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### T5-BASE ROUGE-1: 39.66. Result ID: 698e282affc3901de09b3f2b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### MINILM ROUGE-1: 35.6. Result ID: 698e282affc3901de09b3f2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### BertSumm ROUGE-1: 31.16. Result ID: 698e282affc3901de09b3f2a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### SOTA\* ROUGE-1: 24.58. Result ID: 698e282affc3901de09b3f28. [Reporting paper]() Recorded evaluation configuration: {"category":"Baseline","implementation":"trained on same data"}. This record does not assert independent reproduction. ### Pointer Generator (PG) ROUGE-1: 23.31. Result ID: 698e282affc3901de09b3f27. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Transformer ROUGE-1: 22.25. Result ID: 698e282affc3901de09b3f29. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Seq2Seq + Attention ROUGE-1: 19.45. Result ID: 698e282affc3901de09b3f26. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. ### Seq2Seq ROUGE-1: 17.77. Result ID: 698e282affc3901de09b3f25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"category":"Baseline"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.