Reference-free Conversation Evaluation on CRSArena-Eval KG
0.766Pearson Correlation (r)FACE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.766 | 0.679 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.719 | 0.677 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.672 | 0.557 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.664 | 0.611 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.656 | 0.642 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.655 | 0.557 | |
| UniEvalLevel=Dialogue-level, Aspect=Overall impression2025.05 | 0.618 | 0.659 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.599 | 0.546 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Overall impression2025.05 | 0.599 | 0.521 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.593 | 0.484 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.58 | 0.538 | |
| UniEvalLevel=Dialogue-level, Aspect=Understanding2025.05 | 0.563 | 0.541 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.562 | 0.52 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.543 | 0.527 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.538 | 0.481 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Understanding2025.05 | 0.535 | 0.475 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.533 | 0.505 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.518 | 0.543 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.492 | 0.486 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.485 | 0.463 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.485 | 0.461 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.475 | 0.434 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.471 | 0.453 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.452 | 0.452 | |
| FACEBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.449 | 0.404 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.443 | 0.447 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.435 | 0.397 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.43 | 0.422 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Task completion2025.05 | 0.426 | 0.411 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.425 | 0.378 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.424 | 0.403 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Relevance2025.05 | 0.419 | 0.408 | |
| UniEvalLevel=Turn-level, Aspect=Relevance2025.05 | 0.416 | 0.428 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=All, Aspect=All2025.05 | 0.383 | 0.352 | |
| G-EvalBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.334 | 0.316 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.309 | 0.279 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.308 | 0.324 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Interest arousal2025.05 | 0.297 | 0.322 | |
| UniEvalLevel=Turn-level, Aspect=Interestingness2025.05 | 0.262 | 0.401 | |
| FACE w/o trainBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.24 | 0.419 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.238 | 0.231 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Turn-level, Aspect=Interestingness2025.05 | 0.203 | 0.19 | |
| LLMDirectBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.137 | 0.137 | |
| LLMCoT+ICLBackbone=Llama-3.1-8B-Instruct, Level=Dialogue-level, Aspect=Efficiency2025.05 | 0.114 | 0.114 |