Conversation Summarization on Reddit
55.66QAGSThreadSumm
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ThreadSummLLM Backbone=CLAUDE2026.04 | 55.66 | 42.42 | 34.37 | |
| ThreadSummLLM Backbone=GPT-42026.04 | 50.34 | 30.4 | 33.3 | |
| ThreadSummLLM Backbone=LLAMA2026.04 | 50 | 33.98 | 25.25 | |
| mRedditSummLLM Backbone=CLAUDE2026.04 | 45.66 | 30.64 | 26.82 | |
| CHRONOSLLM Backbone=CLAUDE2026.04 | 45.43 | 32.11 | 26.35 | |
| CHRONOSLLM Backbone=LLAMA2026.04 | 43.09 | 28.82 | 27.1 | |
| CHRONOSLLM Backbone=GPT-42026.04 | 41.49 | 30.31 | 28.59 | |
| arg-graphLLM Backbone=CLAUDE2026.04 | 40.91 | 32.01 | 30.19 | |
| mRedditSummLLM Backbone=LLAMA2026.04 | 40.34 | 30.34 | 30.37 | |
| arg-graphLLM Backbone=GPT-42026.04 | 38.46 | 30.18 | 27.11 | |
| VanillaLLM Backbone=CLAUDE2026.04 | 38.34 | 29.51 | 30.88 | |
| mRedditSummLLM Backbone=GPT-42026.04 | 38.23 | 30.76 | 29.55 | |
| VanillaLLM Backbone=GPT-42026.04 | 36.46 | 30.16 | 30.54 | |
| arg-graphLLM Backbone=LLAMA2026.04 | 35.49 | 30.93 | 24.21 | |
| VanillaLLM Backbone=LLAMA2026.04 | 33.2 | 28.66 | 30.53 | |
| ThreadSummevaluation=LLM-as-a-judge2026.04 | 9.1 | 9.3 | 8.9 | |
| mRedditSummevaluation=LLM-as-a-judge2026.04 | 8.8 | 8 | 8.6 | |
| ThreadSummevaluation=LLM-as-a-judge2026.04 | 8.8 | 8.4 | 8.2 | |
| Vanillaevaluation=LLM-as-a-judge2026.04 | 8.7 | 8.2 | 8.6 | |
| CHRONOSevaluation=LLM-as-a-judge2026.04 | 8.6 | 8.4 | 8.2 | |
| arg-graphevaluation=LLM-as-a-judge2026.04 | 8.2 | 7.8 | 8.4 | |
| mRedditSummevaluation=LLM-as-a-judge2026.04 | 8.2 | 8 | 8.2 | |
| CHRONOSevaluation=LLM-as-a-judge2026.04 | 8 | 7.6 | 7.8 | |
| Vanillaevaluation=LLM-as-a-judge2026.04 | 7.8 | 7.8 | 7.4 | |
| arg-graphevaluation=LLM-as-a-judge2026.04 | 7.6 | 7.8 | 7.2 |