Text Summarization on SummEval Global
85.2CoherenceThemis-8B
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Themis-8BParameters=8B2025.02 | 85.2 | 87.3 | 80.9 | 84.7 | 84.5 | 84 | |
| CompassJudger-32BParameters=32B2025.02 | 84.4 | 89.9 | 79.7 | 88.1 | 85.5 | 86.2 | |
| Qwen-2.5-72BParameters=72B2025.02 | 81.4 | 86.8 | 83.8 | 80 | 83 | 86.9 | |
| Prometheus-2-8x7BParameters=8x7B2025.02 | 78.4 | 82.8 | 71.4 | 72.4 | 76.2 | 79 | |
| GPT-4o2025.02 | 76.5 | 70.8 | 70.2 | 80.1 | 74.4 | 80.7 | |
| Phi-4-14BParameters=14B2025.02 | 76.1 | 72.1 | 77.5 | 75.2 | 75.2 | 81.7 | |
| DeepSeek-V32025.02 | 73.9 | 66.5 | 71 | 76.3 | 71.9 | 77.9 | |
| Prometheus-2-7BParameters=7B2025.02 | 73.9 | 73.3 | 63.9 | 74.6 | 71.4 | 74.6 | |
| GPT-4o mini2025.02 | 73.8 | 72.6 | 71.2 | 83.5 | 75.3 | 80.5 | |
| Gemma-2-27BParameters=27B2025.02 | 73.8 | 78.7 | 69.8 | 79 | 75.3 | 78 | |
| GPT-4 Turbo2025.02 | 73.1 | 69.1 | 69 | 78.5 | 72.4 | 79.5 | |
| CRITIQUELLM-6BParameters=6B2025.02 | 71 | 73.7 | 61.5 | 76.5 | 70.7 | 69.3 | |
| Llama-3.1-70BParameters=70B2025.02 | 70 | 77.9 | 71.2 | 69.3 | 72.1 | 76.8 | |
| GPT-3.5 Turbo2025.02 | 67.3 | 66.7 | 65.3 | 77.9 | 69.3 | 75.1 | |
| Auto-J-13BParameters=13B2025.02 | 61.5 | 61 | 56.9 | 64.4 | 61 | 61.3 | |
| Prometheus-13BParameters=13B2025.02 | 57.6 | 65.3 | 64.5 | 61.3 | 62.2 | 63 |