Persona Simulation Naturalness Evaluation on PersonaChat (test)
0.718CS (Coherence Score)SPC-New
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SPC-NewDescription=Synthetic Persona Chat (New variant)2026.03 | 0.718 | 0.004 | 0.011 | 45.8 | 0.519 | 0.023 | 0.828 | 0.864 | -13.58 | |
| Qwen3Parameters=14B2026.03 | 0.695 | 0.005 | 0.012 | 42.4 | 0.532 | 0.039 | 0.814 | 0.883 | -11.66 | |
| SPCDescription=Synthetic Persona Chat via Generator-Critic2026.03 | 0.682 | 0.007 | 0.017 | 39.4 | 0.57 | 0.034 | 0.804 | 0.897 | -10.29 | |
| Gemma 3Parameters=27B2026.03 | 0.657 | 0.005 | 0.012 | 32.5 | 0.611 | 0.037 | 0.791 | 0.915 | -8.5 | |
| Qwen3Parameters=1.7B2026.03 | 0.643 | 0.006 | 0.017 | 23.5 | 0.661 | 0.026 | 0.782 | 0.927 | -7.27 | |
| Qwen3Parameters=4B2026.03 | 0.618 | 0.004 | 0.012 | 27.1 | 0.694 | 0.035 | 0.768 | 0.946 | -5.35 | |
| Gemma 3Parameters=1B2026.03 | 0.617 | 0.004 | 0.008 | 23.9 | 0.732 | 0.017 | 0.768 | 0.947 | -5.35 | |
| Gemma 3Parameters=12B2026.03 | 0.612 | 0.006 | 0.014 | 25.7 | 0.703 | 0.037 | 0.763 | 0.953 | -4.66 | |
| Gemma 3Parameters=4B2026.03 | 0.608 | 0.004 | 0.014 | 21.7 | 0.709 | 0.038 | 0.761 | 0.956 | -4.39 | |
| Qwen3Parameters=30B2026.03 | 0.601 | 0.004 | 0.011 | 23.2 | 0.736 | 0.031 | 0.758 | 0.96 | -4.02 | |
| PersonaChatType=Reference Baseline2026.03 | 0.555 | 0.006 | 0.017 | 15.7 | 0.796 | 0.047 | 0.729 | 1 | 0 |