Correlation Evaluation on Topical-Chat
0.577Spearman CorrelationUniEval (Dial)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| UniEval (Dial)Training Dataset=Topical-Chat2023.07 | 0.577 | — | |
| DecompEvalType=Instruction-tuned PLM (Zero-shot)2023.07 | 0.499 | — | |
| USR - DR (x = c)component=Dialog Retrieval, context=context2020.05 | 0.4877 | 0.3533 | |
| USR2020.05 | 0.4645 | 0.4555 | |
| USR - DR (x = f)component=Dialog Retrieval, context=fact2020.05 | 0.4468 | 0.222 | |
| USRMetric Category=Reference Free Metrics, Type=Full USR Metric2020.05 | 0.4192 | 0.422 | |
| BERTScoreversion=base2020.05 | 0.4121 | 0.3901 | |
| METEOR2020.05 | 0.3909 | 0.3328 | |
| USR2020.05 | 0.3769 | 0.416 | |
| USR - DR (x = c)component=Dialog Retrieval, context=context2020.05 | 0.365 | 0.3391 | |
| METEORMetric Category=Word-Overlap Metrics2020.05 | 0.3365 | 0.3908 | |
| USR2020.05 | 0.3353 | 0.3175 | |
| USR - MLMcomponent=Masked Language Model2020.05 | 0.3268 | 0.3264 | |
| USR - MLMcomponent=Masked Language Model2020.05 | 0.3254 | 0.337 | |
| USR - DR (x = c)Metric Category=Reference Free Metrics, Context=dialog context (c)2020.05 | 0.3245 | 0.4068 | |
| BERTScoreModel Version=base, Metric Category=Embedding Based Metrics2020.05 | 0.3229 | 0.354 | |
| USR2020.05 | 0.3152 | 0.2932 | |
| UniEval (Summ)Training Dataset=SummEval2023.07 | 0.315 | — | |
| USR - MLMMetric Category=Reference Free Metrics, Sub-metric=Masked Language Modeling2020.05 | 0.3086 | 0.3345 | |
| USR2020.05 | 0.3037 | 0.2763 | |
| METEOR2020.05 | 0.3018 | 0.2495 | |
| BERTScoreModel Version=large, Metric Category=Embedding Based Metrics2020.05 | 0.2982 | 0.3252 | |
| BLEU-2Metric Category=Word-Overlap Metrics2020.05 | 0.2862 | 0.3012 | |
| ROUGE-LMetric Category=Word-Overlap Metrics2020.05 | 0.2745 | 0.287 | |
| BLEU-1Metric Category=Word-Overlap Metrics2020.05 | 0.2728 | 0.2876 | |
| BLEU-3Metric Category=Word-Overlap Metrics2020.05 | 0.2569 | 0.3006 | |
| BERTScoreversion=base2020.05 | 0.2502 | 0.2611 | |
| BLEU-4Metric Category=Word-Overlap Metrics2020.05 | 0.216 | 0.2956 | |
| BERTScoreversion=base2020.05 | 0.2094 | 0.226 | |
| Vector ExtremaMetric Category=Embedding Based Metrics2020.05 | 0.2032 | 0.2091 | |
| Embedding AverageMetric Category=Embedding Based Metrics2020.05 | 0.1803 | 0.2038 | |
| Greedy MatchingMetric Category=Embedding Based Metrics2020.05 | 0.1712 | 0.1943 | |
| F-1Metric Category=Word-Overlap Metrics2020.05 | 0.1645 | 0.169 | |
| USR - DR (x = f)Metric Category=Reference Free Metrics, Context=fact (f)2020.05 | 0.1419 | 0.3221 | |
| Skip-ThoughtMetric Category=Embedding Based Metrics2020.05 | 0.104 | 0.1181 |