Multi-dimensional Summary Evaluation on Deliberation Summary Evaluation Dataset (test)
0.504Representativeness (Pearson)DeliberationJudge
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| DeliberationJudge2025.10 | 0.504 | 0.47 | 0.454 | 0.444 | 0.492 | 0.492 | 0.416 | 0.381 | |
| GPT-5Few-shot demonstrations=5-shot2025.10 | 0.422 | 0.372 | 0.116 | 0.131 | 0.226 | 0.205 | 0.319 | 0.269 | |
| GPT-5-miniFew-shot demonstrations=3-shot2025.10 | 0.399 | 0.347 | 0.094 | 0.095 | 0.241 | 0.221 | 0.344 | 0.293 | |
| GPT-5-miniFew-shot demonstrations=5-shot2025.10 | 0.399 | 0.336 | 0.103 | 0.104 | 0.21 | 0.187 | 0.305 | 0.268 | |
| GPT-5Few-shot demonstrations=3-shot2025.10 | 0.398 | 0.321 | 0.143 | 0.155 | 0.245 | 0.214 | 0.36 | 0.306 | |
| Claude-3.7-sonnetFew-shot demonstrations=3-shot2025.10 | 0.372 | 0.302 | 0.145 | 0.157 | 0.32 | 0.278 | 0.353 | 0.304 | |
| Claude-3.7-sonnetFew-shot demonstrations=5-shot2025.10 | 0.365 | 0.309 | 0.164 | 0.176 | 0.309 | 0.271 | 0.348 | 0.307 |