Role-playing on CharacterBench 1.0 (test)
4.444MCQwen2.5-7B-Instruct + Character-R1
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct + Character-R1Backbone=Qwen2.5-7B, Method=Character-R12026.01 | 4.444 | 2.688 | 4.017 | 4.381 | 4.544 | 3.317 | 3.956 | 3.463 | 3.072 | 4.925 | 4.861 | 3.03 | 3.115 | 3.832 | |
| Llama-3.2-3B-Instruct + Character-R1Backbone=Llama-3.2-3B, Method=Character-R12026.01 | 4.275 | 2.144 | 3.983 | 4.556 | 4.95 | 3.35 | 4.038 | 3.463 | 3.134 | 4.85 | 4.748 | 2.94 | 2.97 | 3.8 | |
| Qwen2.5-7B-RoleMRC2026.01 | 4.175 | 2.231 | 3.775 | 4.3 | 4.713 | 3.225 | 3.688 | 3.188 | 2.927 | 4.75 | 4.735 | 3 | 3.105 | 3.678 | |
| Llama-3.1-8B-RoleMRC2026.01 | 4.169 | 2.163 | 3.717 | 4.231 | 4.588 | 3.225 | 3.606 | 3.125 | 2.864 | 4.8 | 4.748 | 3.04 | 3.06 | 3.641 | |
| Crab2026.01 | 4.131 | 2.175 | 3.6 | 4.25 | 4.65 | 3.167 | 3.681 | 3.331 | 2.965 | 4.85 | 4.684 | 3.05 | 3.11 | 3.665 | |
| Haruhi-Zero-7B2026.01 | 4.088 | 2.15 | 3.583 | 4.256 | 4.525 | 3.192 | 3.613 | 3.456 | 2.965 | 4.85 | 4.773 | 2.99 | 3.065 | 3.654 | |
| Qwen2.5-72B-InstructType=Large Foundation Model2026.01 | 4.06 | 2.558 | 4.102 | 3.222 | 4.531 | 3.439 | 3.917 | 3.398 | 2.962 | 4.897 | 4.815 | 3.512 | 3.418 | 3.756 | |
| Qwen2.5-7B-Instruct + RARBackbone=Qwen2.5-7B, Framework=RAR2026.01 | 4.059 | 2.585 | 3.934 | 4.272 | 4.314 | 3.977 | 4.09 | 3.145 | 2.809 | 4.825 | 4.808 | 2.814 | 2.946 | 3.737 | |
| CoSER-Llama-3.1-8B2026.01 | 4.056 | 2.113 | 3.958 | 4.213 | 4.506 | 3.25 | 3.7 | 3.156 | 2.883 | 4.95 | 4.874 | 3.045 | 2.97 | 3.667 | |
| Qwen2.5-7B-Instruct + SFTBackbone=Qwen2.5-7B, Adaptation=SFT2026.01 | 4.02 | 2.165 | 3.873 | 4.105 | 4.24 | 3.207 | 3.545 | 3.17 | 2.771 | 4.72 | 4.788 | 3 | 3.056 | 3.589 | |
| Humanish-Roleplay2026.01 | 4.013 | 2.35 | 4 | 4.481 | 4.694 | 3.267 | 3.975 | 3.444 | 3.053 | 4.75 | 4.823 | 3.01 | 3.145 | 3.77 | |
| Qwen2.5-7B-Instruct + NeekoBackbone=Qwen2.5-7B, Framework=Neeko2026.01 | 3.997 | 2.192 | 3.775 | 3.288 | 4.257 | 3.084 | 3.212 | 3.229 | 2.693 | 4.55 | 4.703 | 2.799 | 2.942 | 3.44 | |
| Claude-4-opusType=Large Foundation Model2026.01 | 3.997 | 2.435 | 4.398 | 4.362 | 4.508 | 3.751 | 3.877 | 3.587 | 3.176 | 4.936 | 4.735 | 3.151 | 3.351 | 3.866 | |
| Gemini-3-proType=Large Foundation Model2026.01 | 3.946 | 2.905 | 4.099 | 4.133 | 4.167 | 3.529 | 3.671 | 3.319 | 2.973 | 4.82 | 4.633 | 3.613 | 3.423 | 3.787 | |
| Llama-3-70B-InstructType=Large Foundation Model2026.01 | 3.94 | 2.59 | 3.95 | 3.96 | 4.39 | 3.35 | 3.33 | 3.06 | 2.89 | 4.71 | 4.74 | 3.4 | 3.51 | 3.678 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B2026.01 | 3.938 | 2.356 | 3.8 | 4.275 | 4.631 | 3.233 | 3.856 | 3.238 | 3.097 | 4.918 | 4.861 | 3.12 | 3.12 | 3.726 | |
| Llama-3.2-3B-Instruct + RARBackbone=Llama-3.2-3B, Framework=RAR2026.01 | 3.848 | 2.01 | 3.978 | 4.373 | 4.511 | 4.112 | 3.918 | 3.24 | 2.91 | 4.785 | 4.743 | 2.718 | 2.964 | 3.701 | |
| Llama-3.2-3B-Instruct + NeekoBackbone=Llama-3.2-3B, Framework=Neeko2026.01 | 3.796 | 1.855 | 3.816 | 3.428 | 4.513 | 3.192 | 3.416 | 3.319 | 2.747 | 4.561 | 4.582 | 2.665 | 2.972 | 3.451 | |
| GPT-4oType=Large Foundation Model2026.01 | 3.793 | 2.647 | 3.978 | 4.027 | 4.484 | 3.414 | 3.723 | 3.046 | 2.974 | 4.763 | 4.771 | 3.261 | 3.51 | 3.722 | |
| CharacterGLMType=Large Foundation Model2026.01 | 3.76 | 2.18 | 3.97 | 3.8 | 4.03 | 2.89 | 3.26 | 2.94 | 2.64 | 4.53 | 4.51 | 3.16 | 3.32 | 3.461 | |
| Baichuan-NPCType=Large Foundation Model2026.01 | 3.672 | 2.134 | 4.132 | 4.216 | 4.254 | 3.366 | 4.022 | 3.001 | 3.179 | 4.83 | 4.897 | 2.975 | 3.297 | 3.69 | |
| Llama-3.2-3B-Instruct + SFTBackbone=Llama-3.2-3B, Adaptation=SFT2026.01 | 3.606 | 2.063 | 3.892 | 4.169 | 4.55 | 3.5 | 3.894 | 3.219 | 3.003 | 4.888 | 4.823 | 3.35 | 3.31 | 3.713 | |
| Llama-3.2-3B-InstructBackbone=Llama-3.2-3B2026.01 | 3.588 | 2.088 | 3.892 | 4.3 | 4.606 | 3.3 | 4.025 | 3.306 | 3.053 | 4.84 | 4.649 | 2.89 | 3.32 | 3.681 | |
| GLM-4Type=Large Foundation Model2026.01 | 3.524 | 2.373 | 3.701 | 4.103 | 4.38 | 3.487 | 3.728 | 3.13 | 2.987 | 4.826 | 4.876 | 3.208 | 3.5 | 3.679 | |
| GPT-3.5-turboType=Large Foundation Model2026.01 | 3.49 | 2.451 | 3.692 | 4.155 | 4.345 | 3.56 | 3.635 | 3.09 | 2.838 | 4.735 | 4.761 | 3.619 | 3.758 | 3.702 | |
| CharacterGLM-6B2026.01 | 3.245 | 2.1 | 3.543 | 3.41 | 3.365 | 3.1 | 3.07 | 2.61 | 2.5 | 4.48 | 4.8 | 2.84 | 2.7 | 3.213 | |
| Peach-9B-8k-Roleplay2026.01 | 2.931 | 2.344 | 3.717 | 3.388 | 3.8 | 3.292 | 3.3 | 2.863 | 2.802 | 4.838 | 4.798 | 2.475 | 2.435 | 3.306 | |
| Peach-2.0-9B-8k-Roleplay2026.01 | 2.819 | 2.4 | 3.425 | 4.263 | 4.4 | 3.867 | 4.088 | 3.35 | 3.028 | 4.475 | 4.47 | 3.485 | 3.68 | 3.673 |