Reference-free Conversation Evaluation on CRSArena-Eval RD
0.712Pearson Correlation (r)FACE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.712 | 0.668 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.65 | 0.635 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.617 | 0.555 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.605 | 0.574 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.577 | 0.577 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.57 | 0.453 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.564 | 0.522 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.551 | 0.53 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.549 | 0.55 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.522 | 0.482 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.498 | 0.472 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.49 | 0.471 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.49 | 0.444 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.488 | 0.482 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.484 | 0.534 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.482 | 0.392 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.481 | 0.457 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.468 | 0.462 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.464 | 0.455 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.453 | 0.446 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.447 | 0.43 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.443 | 0.437 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.433 | 0.422 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.432 | 0.422 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.425 | 0.4 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.405 | 0.363 | |
| UniEvalLevel=Dialogue-level, Aspect=Overall impression2025.05 | 0.395 | 0.387 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.36 | 0.341 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.351 | 0.364 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.342 | 0.329 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.339 | 0.423 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.332 | 0.325 | |
| UniEvalLevel=Turn-level, Aspect=Relevance2025.05 | 0.311 | 0.288 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.302 | 0.289 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.279 | 0.29 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.248 | 0.26 | |
| UniEvalLevel=Dialogue-level, Aspect=Understanding2025.05 | 0.246 | 0.225 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.235 | 0.255 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.217 | 0.203 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.188 | 0.174 | |
| UniEvalLevel=Turn-level, Aspect=Interestingness2025.05 | 0.182 | 0.242 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.175 | 0.177 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.174 | 0.174 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.101 | 0.101 |