Role-playing Dialogue Evaluation on RoleChat gold-standard evaluation set
96.6Logical CoherenceGPT-4.1
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4.1Modality=Text-Modality, Model Access Type=Closed-Source Models2026.04 | 96.6 | 92.1 | 28.4 | 19.5 | 48.2 | 56.96 | 100 | |
| RoleJudgeModality=Multi-Modality2026.04 | 94.8 | 90.2 | 85.1 | 75.9 | 84 | 86 | 100 | |
| Gemini3 ProModality=Multi-Modality, Model Access Type=Closed-Source Models2026.04 | 86.5 | 72.9 | 75.8 | 51.6 | 62.2 | 69.8 | 100 | |
| Qwen3-32BModality=Text-Modality, Model Access Type=Open-Source Models2026.04 | 66.2 | 46.5 | 21.2 | 14.6 | 35.4 | 36.78 | 94.3 | |
| GPT-4o-audioModality=Multi-Modality, Model Access Type=Closed-Source Models2026.04 | 65.2 | 42.3 | 61.2 | 52.4 | 44.2 | 53.06 | 94.2 | |
| Qwen3-OmniModality=Multi-Modality, Model Access Type=Open-Source Models2026.04 | 63.8 | 43.3 | 51.6 | 22.1 | 35.5 | 43.26 | 75.8 | |
| Qwen3-8BModality=Text-Modality, Model Access Type=Open-Source Models2026.04 | 62.5 | 42.1 | 18.5 | 12.3 | 32.1 | 33.5 | 91.1 | |
| Qwen2-AudioModality=Multi-Modality, Model Access Type=Open-Source Models2026.04 | 40.9 | 25.1 | 42.1 | 11.1 | 34.1 | 30.66 | 10.2 | |
| Qwen-AudioModality=Multi-Modality, Model Access Type=Open-Source Models2026.04 | 35.2 | 29.3 | 34.2 | 16.2 | 32.3 | 29.48 | 0 | |
| SALMONN-7BModality=Multi-Modality, Model Access Type=Open-Source Models2026.04 | 11.2 | 23.2 | 43.2 | 12.1 | 22.1 | 22.36 | 6.2 |