Summarization Faithfulness on ArXiv
0.4574SummaCExtract-to-Abstract (E2A)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Extract-to-Abstract (E2A)System=E2A, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.4574 | 0.1212 | |
| o1System=o1, Model Class=Large Reasoning Models (LRM), Shots=0-shot2025.12 | 0.4423 | 0.3171 | |
| Question-Answer Guided (QAG)System=QAG, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.409 | 0.2966 | |
| Chain-of-Thought (CoT)System=COT, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.4054 | 0.0746 | |
| Cited Summarization (Cite)System=Cite, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.3197 | 0.6019 | |
| GPT-5System=GPT-5, Model Class=Large Reasoning Models (LRM), Shots=0-shot2025.12 | 0.2738 | 0.563 | |
| VanillaSystem=Vanilla, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.2524 | 0.5625 | |
| Iterative Refine (IR)System=IR, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.2517 | 0.5472 | |
| o3System=o3, Model Class=Large Reasoning Models (LRM), Shots=0-shot2025.12 | 0.2505 | 0.4863 | |
| Decomposition (Deco)System=Deco, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.2461 | 0.5542 | |
| Self-Consistency (SC)System=SC, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.2425 | 0.5312 | |
| Plan-then-Write (Plan)System=Plan, Model Class=LLM (GPT-4.1), Shots=0-shot2025.12 | 0.2385 | 0.5664 |