Summarization on WikiHow (test)
19.23ROUGEVanilla
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| VanillaBackbone=GPT-4.1, Shot-setting=2-shot2025.12 | 19.23 | — | — | — | — | — | 85.52 | |
| Chain-of-Thought (CoT)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 19.2 | — | — | — | — | — | 85.46 | |
| Self-Consistency (SC)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 18.71 | — | — | — | — | — | 84.83 | |
| Cited Summarization (Cite)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 18.66 | — | — | — | — | — | 84.05 | |
| Iterative Refine (IR)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 18.4 | — | — | — | — | — | 85.2 | |
| Extract-to-Abstract (E2A)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 18.32 | — | — | — | — | — | 85.4 | |
| o1Backbone=Large Reasoning Model, Shot-setting=2-shot2025.12 | 17.38 | — | — | — | — | — | 83.52 | |
| Decomposition (Deco)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 17.16 | — | — | — | — | — | 85.26 | |
| GPT-5Backbone=Large Reasoning Model, Shot-setting=2-shot2025.12 | 16.41 | — | — | — | — | — | 84.06 | |
| o3Backbone=Large Reasoning Model, Shot-setting=2-shot2025.12 | 16.31 | — | — | — | — | — | 84.61 | |
| Question-Answer Guided (QAG)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 16.25 | — | — | — | — | — | 84.39 | |
| Plan-then-Write (Plan)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 15.42 | — | — | — | — | — | 84.59 | |
| BART-PTShots=100-shot2022.11 | — | 4.3 | 4 | 3.93 | 4.17 | — | — | |
| Goldtype=human reference2022.11 | — | 4.72 | 4.57 | 4.87 | 4.88 | — | — | |
| GPT-3.5Model=text-davinci-002, Shots=1-shot2022.11 | — | 4.9 | 4.83 | 4.63 | 4.58 | — | — | |
| PEGASUSShots=100-shot2022.11 | — | 4.43 | 4.17 | 4.17 | 4.33 | — | — | |
| PEGASUS_LARGENumber of Parameters=470M2022.08 | — | — | — | — | — | 18.5 | — | |
| Prior SOTA2022.08 | — | — | — | — | — | 18.5 | — | |
| UNISUMMShots=100-shot2022.11 | — | 4.68 | 4.43 | 4.67 | 4.67 | — | — | |
| Z-Code++_LARGENumber of Parameters=710M2022.08 | — | — | — | — | — | 22.1 | — |