Summarization Faithfulness Evaluation on 100 articles (Protocol 1)
19Llama-70B Faithfulness (%)Claude-3.5
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Claude-3.5Generation Protocol=Protocol 1 (Unconstrained Generation)2026.06 | 19 | 6 | 3 | 2 | 21 | 10.2 | |
| GPT-4oGeneration Protocol=Protocol 1 (Unconstrained Generation)2026.06 | 19 | 3 | 1 | 0 | 30 | 10.6 | |
| Llama-70BGeneration Protocol=Protocol 1 (Unconstrained Generation)2026.06 | 17 | 5 | 1 | 2 | 20 | 9 | |
| Jais-70BGeneration Protocol=Protocol 1 (Unconstrained Generation)2026.06 | 12 | 7 | 7 | 3 | 5 | 6.8 | |
| Qwen2-72BGeneration Protocol=Protocol 1 (Unconstrained Generation)2026.06 | 9 | 2 | 1 | 0 | 15 | 5.4 | |
| ALLaMGeneration Protocol=Protocol 1 (Unconstrained Generation)2026.06 | 3 | 0 | 0 | 0 | 3 | 1.2 |