Audio-Grounded Character Role-playing on AudioRole-Demo
4AP RankARP-Model
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ARP-Model2025.09 | 4 | 2.75 | 100 | 90 | — | — | |
| MiniCPM-o-2.62025.09 | 3 | 2.56 | 100 | 92 | — | — | |
| GPT-4o-Audio2025.09 | 1.61 | 3.58 | 84 | 76 | — | — | |
| GLM-4-Voice2025.09 | 1.39 | 1.11 | 84 | 88 | — | — | |
| ARP-ModelCharacter=Sheldon2025.09 | 0.42 | 0.44 | — | — | 6.7 | 0.28 | |
| ARP-ModelCharacter=Buffy2025.09 | 0.39 | 0.36 | — | — | 6.7 | 0.43 | |
| ARP-ModelCharacter=Laura Roslin2025.09 | 0.33 | 0.41 | — | — | 6.2 | 0.33 | |
| ARP-ModelCharacter=Avg.2025.09 | 0.31 | 0.36 | — | — | 6.5 | 0.39 | |
| GPT-4o AudioCharacter=Sheldon2025.09 | 0.29 | 1 | — | — | 7.7 | 1.3 | |
| ARP-ModelCharacter=Walter White2025.09 | 0.26 | 0.17 | — | — | 6.4 | 0.5 | |
| ARP-ModelCharacter=Tyrion Lannister2025.09 | 0.25 | 0.26 | — | — | 6.8 | 0.55 | |
| MiniCPM-o 2.6Character=Laura Roslin2025.09 | 0.24 | 0.41 | — | — | 7.1 | 0.71 | |
| MiniCPM-o 2.6Character=Sheldon2025.09 | 0.24 | 0.54 | — | — | 6.8 | 1.1 | |
| ARP-ModelCharacter=Jack Bauer2025.09 | 0.23 | 0.49 | — | — | 6 | 0.22 | |
| MiniCPM-o 2.6Character=Walter White2025.09 | 0.22 | 0.38 | — | — | 7.1 | 0.7 | |
| MiniCPM-o 2.6Character=Avg.2025.09 | 0.17 | 0.35 | — | — | 6.9 | 0.74 | |
| MiniCPM-o 2.6Character=Tyrion Lannister2025.09 | 0.14 | 0.14 | — | — | 6.9 | 0.59 | |
| MiniCPM-o 2.6Character=Jack Bauer2025.09 | 0.13 | 0.3 | — | — | 6.5 | 0.66 | |
| GLM-4-VoiceCharacter=Buffy2025.09 | 0.1 | 0.25 | — | — | 7.6 | 0.52 | |
| MiniCPM-o 2.6Character=Buffy2025.09 | 0.09 | 0.34 | — | — | 6.8 | 0.69 | |
| GPT-4o AudioCharacter=Walter White2025.09 | 0.08 | 1.3 | — | — | 7.5 | 1.6 | |
| GLM-4-VoiceCharacter=Walter White2025.09 | 0.07 | 0.43 | — | — | 7.6 | 0.37 | |
| GPT-4o AudioCharacter=Avg.2025.09 | 0.06 | 1.1 | — | — | 7.7 | 1.7 | |
| GLM-4-VoiceCharacter=Jack Bauer2025.09 | 0.05 | 0.24 | — | — | 7.5 | 0.37 | |
| GLM-4-VoiceCharacter=Avg.2025.09 | 0.05 | 0.26 | — | — | 7.6 | 0.43 | |
| GLM-4-VoiceCharacter=Tyrion Lannister2025.09 | 0.04 | 0.1 | — | — | 7.6 | 0.36 | |
| GLM-4-VoiceCharacter=Laura Roslin2025.09 | 0.03 | 0.24 | — | — | 7.5 | 0.5 | |
| GPT-4o AudioCharacter=Jack Bauer2025.09 | 0.02 | 1.3 | — | — | 7.8 | 1.9 | |
| GPT-4o AudioCharacter=Buffy2025.09 | 0.01 | 1.1 | — | — | 7.7 | 1.8 | |
| GLM-4-VoiceCharacter=Sheldon2025.09 | 0.01 | 0.32 | — | — | 7.6 | 0.43 | |
| GPT-4o AudioCharacter=Tyrion Lannister2025.09 | 0 | 0.95 | — | — | 7.6 | 1.9 | |
| GPT-4o AudioCharacter=Laura Roslin2025.09 | -0.02 | 0.8 | — | — | 7.7 | 1.8 |