Summarization on SummEval (Kendall's Tau, Completeness, Groundedness)
0.72CompletenessJury-on-Demand
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Jury-on-DemandJury Configuration=Jury-on-Demand2025.12 | 0.72 | 0.71 | 0.61 | — | |
| Gemini 2.5 FlashModel Variant=Gemini 2.5 Flash2025.12 | 0.68 | — | 0.61 | — | |
| DeepSeek R1Model Variant=DeepSeek R12025.12 | 0.68 | — | 0.12 | — | |
| GPT-OSS 120BModel Variant=GPT-OSS 120B2025.12 | 0.67 | — | 0.6 | — | |
| Gemini 2.0 FlashModel Variant=Gemini 2.0 Flash2025.12 | 0.63 | — | 0.56 | — | |
| Gemini 2.5 ProModel Variant=Gemini 2.5 Pro2025.12 | 0.58 | — | 0.64 | — | |
| Claude 3.7Model Variant=Claude 3.72025.12 | 0.57 | — | 0.63 | — | |
| Gemma 3Model Variant=Gemma 32025.12 | 0.33 | — | 0.59 | — | |
| Phi 4Model Variant=Phi 42025.12 | 0.27 | — | 0.17 | — | |
| LLAMA 3.2Model Variant=LLAMA 3.22025.12 | 0.12 | — | 0.21 | — | |
| GPT-OSS 20BModel Variant=GPT-OSS 20B2025.12 | -0.1 | — | 0.56 | — | |
| Static JuryJury Configuration=Average-All2025.12 | — | 0.7 | — | — | |
| Static JuryJury Configuration=Average-TopK2025.12 | — | 0.72 | — | — | |
| Static JuryJury Configuration=Weighted-Regression2025.12 | — | 0.69 | — | — | |
| Static JuryJury Configuration=Weighted-Tau2025.12 | — | 0.7 | — | — |