Summarization on ArXiv (test) (Comp/Conc/Fact Metrics)
5Completeness ScoreVanilla
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| VanillaBase Model=GPT-4.12025.12 | 5 | 4.29 | 5 | |
| Chain-of-Thought (CoT)Base Model=GPT-4.12025.12 | 5 | 4.05 | 5 | |
| Iterative Refine (IR)Base Model=GPT-4.12025.12 | 5 | 4.69 | 5 | |
| o3Model Category=Large Reasoning Models (LRM)2025.12 | 5 | 4.87 | 5 | |
| GPT-5Model Category=Large Reasoning Models (LRM)2025.12 | 5 | 4.33 | 5 | |
| VanillaModel Category=LLM (GPT-4.1)2025.12 | 5 | 4.29 | 5 | |
| COTModel Category=LLM (GPT-4.1)2025.12 | 5 | 4.05 | 5 | |
| IRModel Category=LLM (GPT-4.1)2025.12 | 5 | 4.69 | 5 | |
| o3Model Category=Large Reasoning Models (LRM)2025.12 | 5 | 4.87 | 5 | |
| GPT-5Model Category=Large Reasoning Models (LRM)2025.12 | 5 | 4.33 | 5 | |
| Self-Consistency (SC)Base Model=GPT-4.12025.12 | 4.99 | 4.68 | 5 | |
| SCModel Category=LLM (GPT-4.1)2025.12 | 4.99 | 4.68 | 5 | |
| Decomposition (Deco)Base Model=GPT-4.12025.12 | 4.98 | 3.91 | 4.97 | |
| Plan-then-Write (Plan)Base Model=GPT-4.12025.12 | 4.98 | 4.38 | 5 | |
| DecoModel Category=LLM (GPT-4.1)2025.12 | 4.98 | 3.91 | 4.97 | |
| PlanModel Category=LLM (GPT-4.1)2025.12 | 4.98 | 4.38 | 5 | |
| o1Model Category=Large Reasoning Models (LRM)2025.12 | 4.97 | 4.88 | 5 | |
| o1Model Category=Large Reasoning Models (LRM)2025.12 | 4.97 | 4.88 | 5 | |
| Cited Summarization (Cite)Base Model=GPT-4.12025.12 | 4.81 | 4.86 | 4.98 | |
| CiteModel Category=LLM (GPT-4.1)2025.12 | 4.81 | 4.86 | 4.98 | |
| Extract-to-Abstract (E2A)Base Model=GPT-4.12025.12 | 4.67 | 4.92 | 4.99 | |
| E2AModel Category=LLM (GPT-4.1)2025.12 | 4.67 | 4.92 | 4.99 | |
| Question-Answer Guided (QAG)Base Model=GPT-4.12025.12 | 3.83 | 3.56 | 4.97 | |
| QA-GModel Category=LLM (GPT-4.1)2025.12 | 3.83 | 3.56 | 4.97 |