Role-playing on CharacterBench
4.043Overall Average ScoreCRPO
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRPOBackbone=Qwen3-8B2026.05 | 4.043 | 4.525 | 2.581 | 4.308 | 4.994 | 4.781 | 4.469 | 3.717 | 3.819 | 3.354 | 4.925 | 4.659 | 3.3 | 3.13 | — | — | — | — | — | — | |
| PPOBackbone=Qwen3-8B2026.05 | 3.901 | 4.306 | 2.244 | 4.2 | 4.844 | 4.569 | 4.169 | 3.583 | 3.575 | 3.091 | 4.85 | 4.722 | 3.335 | 3.225 | — | — | — | — | — | — | |
| GSPOBackbone=Qwen3-8B2026.05 | 3.894 | 4.344 | 2.481 | 4.125 | 4.575 | 4.55 | 4.169 | 3.608 | 3.684 | 3.172 | 4.75 | 4.659 | 3.255 | 3.255 | — | — | — | — | — | — | |
| OARBackbone=Qwen3-8B2026.05 | 3.882 | 4.45 | 2.369 | 4.092 | 4.944 | 4.588 | 4.125 | 3.333 | 3.475 | 3.21 | 4.913 | 4.849 | 3.04 | 3.075 | — | — | — | — | — | — | |
| Character-R1Backbone=Qwen3-8B2026.05 | 3.878 | 4.294 | 2.488 | 4.175 | 4.763 | 4.563 | 4.125 | 3.442 | 3.494 | 3.191 | 4.863 | 4.785 | 3.145 | 3.09 | — | — | — | — | — | — | |
| GPT-4-ChatModel=GPT-4-Chat2026.06 | 3.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.52 | 3.22 | 4.21 | 3.21 | 4.91 | 3.63 | |
| Claude-3-opus2026.05 | 3.866 | 3.997 | 2.435 | 4.398 | 4.508 | 4.362 | 3.877 | 3.751 | 3.587 | 3.176 | 4.936 | 4.735 | 3.151 | 3.351 | — | — | — | — | — | — | |
| GDPOBackbone=Qwen3-8B2026.05 | 3.86 | 4.425 | 2.4 | 4 | — | 4.55 | 4.075 | 3.367 | 3.594 | 3.147 | 4.875 | 4.785 | 3.1 | 3.09 | — | — | — | — | — | — | |
| GPT-4o-miniModel=GPT-4o-mini2026.06 | 3.86 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.72 | 3.18 | 4.24 | 3.23 | 4.9 | 3.45 | |
| Llama-3.3-70B-InstructModel=Llama-3.3-70B-Instruct2026.06 | 3.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.71 | 3.38 | 4.18 | 3.23 | 4.91 | 3.39 | |
| Doubao-1.5-Pro-CharacterModel=Doubao-1.5-Pro-Character2026.06 | 3.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.83 | 3.41 | 4.06 | 3.18 | 4.88 | 3.42 | |
| CRPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.832 | 4.356 | 2.113 | 4.175 | 4.988 | 4.619 | 4.038 | 3.483 | 3.581 | 3.166 | 4.888 | 4.873 | 2.75 | 2.785 | — | — | — | — | — | — | |
| GRPOBackbone=Qwen3-8B2026.05 | 3.829 | 3.913 | 2.456 | 4.125 | 4.194 | 4.413 | 4.094 | 3.617 | 3.444 | 3.097 | 4.863 | 4.861 | 3.405 | 3.295 | — | — | — | — | — | — | |
| Character-R1Backbone=Llama-3.2-3B-Instruct2026.05 | 3.8 | 4.275 | 2.144 | 3.983 | 4.95 | 4.556 | 4.038 | 3.35 | 3.463 | 3.134 | 4.85 | 4.748 | 2.94 | 2.97 | — | — | — | — | — | — | |
| Qwen2-7B-InstructModel=Qwen2-7B-Instruct2026.06 | 3.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.94 | 2.99 | 4.22 | 3 | 4.91 | 3.43 | |
| DAPOBackbone=Qwen3-8B2026.05 | 3.794 | 4.125 | 2.444 | 4.1 | 4.25 | 4.413 | 4.031 | 3.208 | 3.488 | 3.172 | 4.9 | 4.773 | 3.27 | 3.15 | — | — | — | — | — | — | |
| Llama-3.1-8B-InstructModel=Llama-3.1-8B-Instruct2026.06 | 3.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.65 | 3.16 | 4.12 | 3.25 | 4.84 | 3.36 | |
| Gemini-pro2026.05 | 3.787 | 3.946 | 2.905 | 4.099 | 4.167 | 4.133 | 3.671 | 3.529 | 3.319 | 2.973 | 4.82 | 4.633 | 3.613 | 3.423 | — | — | — | — | — | — | |
| REINFORCE++Backbone=Qwen3-8B2026.05 | 3.783 | 4.1 | 2.419 | 4.092 | 4.7 | 4.344 | 3.806 | 3.217 | 3.213 | 2.783 | 4.9 | 4.899 | 3.5 | 3.211 | — | — | — | — | — | — | |
| GRPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.781 | 4.35 | 2.238 | 4.067 | 4.813 | 4.431 | 3.925 | 3.217 | 3.419 | 3.115 | 4.868 | 4.861 | 2.925 | 2.925 | — | — | — | — | — | — | |
| RARBackbone=Qwen3-8B2026.05 | 3.779 | 3.892 | 2.537 | 4.129 | 4.485 | 4.32 | 4.342 | 3.646 | 3.135 | 2.781 | 4.873 | 4.693 | 3.15 | 3.144 | — | — | — | — | — | — | |
| Dr.GRPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.777 | 4.263 | 2.194 | 4.1 | 4.85 | 4.456 | 3.931 | 3.283 | 3.513 | 3.078 | 4.85 | 4.71 | 2.925 | 2.945 | — | — | — | — | — | — | |
| Humanish-Roleplay2026.05 | 3.77 | 4.013 | 2.35 | 4 | 4.694 | 4.481 | 3.975 | 3.267 | 3.444 | 3.053 | 4.75 | 4.823 | 3.01 | 3.145 | — | — | — | — | — | — | |
| Qwen2.5-14B-InstructModel=Qwen2.5-14B-Instruct2026.06 | 3.77 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.02 | 3.11 | 4.06 | 3.26 | 4.93 | 3.13 | |
| OARBackbone=Llama-3.2-3B-Instruct2026.05 | 3.762 | 4.206 | 2.175 | 4.175 | 4.844 | 4.369 | 3.844 | 3.167 | 3.581 | 3.097 | 4.868 | 4.736 | 2.875 | 2.97 | — | — | — | — | — | — | |
| GSPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.76 | 4.238 | 2.156 | 4.15 | 4.838 | 4.45 | 3.806 | 3.208 | 3.444 | 3.041 | 4.825 | 4.811 | 2.975 | 2.94 | — | — | — | — | — | — | |
| Dr.GRPOBackbone=Qwen3-8B2026.05 | 3.759 | 3.869 | 2.574 | 4.092 | 3.681 | 4.406 | 4.131 | 3.6 | 3.625 | 3.153 | 4.825 | 4.76 | 3.05 | 3.1 | — | — | — | — | — | — | |
| Qwen2.5-72B-Instruct2026.05 | 3.756 | 4.06 | 2.558 | 4.102 | 4.531 | 3.222 | 3.917 | 3.439 | 3.398 | 2.962 | 4.897 | 4.815 | 3.512 | 3.418 | — | — | — | — | — | — | |
| GDPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.751 | 4.319 | 2.263 | 3.933 | 4.85 | 4.3 | 3.8 | 3.233 | 3.475 | 3.134 | 4.868 | 4.886 | 2.85 | 2.855 | — | — | — | — | — | — | |
| Qwen2.5-7B-Instruct + Psy-CoT + RAPOBackbone=Qwen2.5-7B-Instruct, Chain-of-Thought=Psy-CoT, Optimization=RAPO2026.06 | 3.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.77 | 3.03 | 4.07 | 3.07 | 4.92 | 3.34 | |
| DAPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.745 | 4.25 | 2.163 | 4.112 | 4.756 | 4.375 | 3.85 | 3.308 | 3.4 | 3.034 | 4.8 | 4.786 | 2.915 | 2.94 | — | — | — | — | — | — | |
| RLOOBackbone=Qwen3-8B2026.05 | 3.731 | 3.556 | 2.425 | 3.792 | 4.519 | 4.263 | 3.931 | 3.717 | 3.119 | 2.852 | 4.788 | 4.773 | 3.47 | 3.295 | — | — | — | — | — | — | |
| GPT-4o2026.05 | 3.722 | 3.793 | 2.647 | 3.978 | 4.484 | 4.027 | 3.723 | 3.414 | 3.046 | 2.974 | 4.763 | 4.771 | 3.261 | 3.51 | — | — | — | — | — | — | |
| SFTBackbone=Llama-3.2-3B-Instruct2026.05 | 3.713 | 3.606 | 2.063 | 3.892 | 4.55 | 4.169 | 3.894 | 3.5 | 3.219 | 3.003 | 4.888 | 4.823 | 3.35 | 3.31 | — | — | — | — | — | — | |
| Llama-3.1-70B-CoSERModel=Llama-3.1-70B-CoSER2026.06 | 3.71 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.85 | 3.18 | 3.92 | 3.17 | 4.81 | 3.19 | |
| GPT-3.5-turbo2026.05 | 3.702 | 3.49 | 2.451 | 3.692 | 4.345 | 4.155 | 3.635 | 3.56 | 3.09 | 2.838 | 4.735 | 4.761 | 3.619 | 3.758 | — | — | — | — | — | — | |
| Qwen3-8B2026.05 | 3.701 | 3.594 | 2.4 | 3.758 | 4.556 | 4.188 | 3.819 | 3.692 | 3.094 | 2.808 | 4.8 | 4.684 | 3.445 | 3.27 | — | — | — | — | — | — | |
| RARBackbone=Llama-3.2-3B-Instruct2026.05 | 3.701 | 3.848 | 2.01 | 3.978 | 4.511 | 4.373 | 3.918 | 4.112 | 3.24 | 2.91 | 4.785 | 4.743 | 2.718 | 2.964 | — | — | — | — | — | — | |
| Qwen2.5-7B-Instruct + Psy-CoT + DAPOBackbone=Qwen2.5-7B-Instruct, Chain-of-Thought=Psy-CoT, Optimization=DAPO2026.06 | 3.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.67 | 3.08 | 3.95 | 3.04 | 4.94 | 3.25 | |
| NeekoBackbone=Qwen3-8B2026.05 | 3.696 | 3.991 | 2.202 | 3.83 | 4.709 | 4.132 | 3.725 | 3.439 | 3.28 | 2.83 | 4.853 | 4.767 | 3.112 | 3.182 | — | — | — | — | — | — | |
| ReMaxBackbone=Qwen3-8B2026.05 | 3.693 | 3.781 | 2.425 | 3.792 | 4.45 | 4.206 | 3.638 | 3.583 | 2.963 | 2.783 | 4.875 | 4.76 | 3.475 | 3.275 | — | — | — | — | — | — | |
| Baichuan-NPC2026.05 | 3.69 | 3.672 | 2.134 | 4.132 | 4.254 | 4.216 | 4.022 | 3.366 | 3.001 | 3.179 | 4.83 | 4.897 | 2.975 | 3.297 | — | — | — | — | — | — | |
| Llama-3.2-3B-Instruct2026.05 | 3.681 | 3.588 | 2.088 | 3.892 | 4.606 | 4.3 | 4.025 | 3.3 | 3.306 | 3.053 | 4.84 | 4.649 | 2.89 | 3.32 | — | — | — | — | — | — | |
| Qwen2.5-7B-Instruct + Psy-CoT + GRPOBackbone=Qwen2.5-7B-Instruct, Chain-of-Thought=Psy-CoT, Optimization=GRPO2026.06 | 3.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.77 | 3.02 | 3.98 | 2.95 | 4.92 | 3.24 | |
| Qwen2.5-7B-Instruct-AdaMARPModel=Qwen2.5-7B-Instruct-AdaMARP2026.06 | 3.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.03 | 3 | 3.99 | 3.07 | 4.8 | 3.04 | |
| GLM-42026.05 | 3.679 | 3.524 | 2.373 | 3.701 | 4.38 | 4.103 | 3.728 | 3.487 | 3.13 | 2.987 | 4.826 | 4.876 | 3.208 | 3.5 | — | — | — | — | — | — | |
| Qwen2.5-7B-RoleMRC2026.05 | 3.678 | 4.175 | 2.231 | 3.775 | 4.713 | 4.3 | 3.688 | 3.225 | 3.188 | 2.927 | 4.75 | 4.735 | 3 | 3.105 | — | — | — | — | — | — | |
| Llama-3-70B-Instruct2026.05 | 3.678 | 3.94 | 2.59 | 3.95 | 4.39 | 3.96 | 3.33 | 3.35 | 3.06 | 2.89 | 4.71 | 4.74 | 3.4 | 3.51 | — | — | — | — | — | — | |
| SFTBackbone=Qwen3-8B2026.05 | 3.673 | 4.025 | 2.219 | 3.925 | 4.731 | 4.331 | 3.65 | 3.208 | 3.25 | 2.795 | 4.8 | 4.71 | 3.03 | 3.075 | — | — | — | — | — | — | |
| Peach-2.0-9B-8k-Roleplay2026.05 | 3.673 | 2.819 | 2.4 | 3.425 | 4.4 | 4.263 | 4.088 | 3.867 | 3.35 | 3.028 | 4.475 | 4.47 | 3.485 | 3.68 | — | — | — | — | — | — | |
| CoSER-Llama-3.1-8B2026.05 | 3.667 | 4.056 | 2.113 | 3.958 | 4.506 | 4.213 | 3.7 | 3.25 | 3.156 | 2.883 | 4.95 | 4.874 | 3.045 | 2.97 | — | — | — | — | — | — | |
| Crab2026.05 | 3.665 | 4.131 | 2.175 | 3.6 | 4.65 | 4.25 | 3.681 | 3.167 | 3.331 | 2.965 | 4.85 | 4.684 | 3.05 | 3.11 | — | — | — | — | — | — | |
| Haruhi-Zero-7B2026.05 | 3.654 | 4.088 | 2.15 | 3.583 | 4.525 | 4.256 | 3.613 | 3.192 | 3.456 | 2.965 | 4.85 | 4.773 | 2.99 | 3.065 | — | — | — | — | — | — | |
| DeepSeek-V3-FlashModel=DeepSeek-V3-Flash2026.06 | 3.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.18 | 3.27 | 3.87 | 2.99 | 4.83 | 3.31 | |
| Qwen2.5-7B-Instruct + Psy-CoT + GSPOBackbone=Qwen2.5-7B-Instruct, Chain-of-Thought=Psy-CoT, Optimization=GSPO2026.06 | 3.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.77 | 2.98 | 3.91 | 2.93 | 4.9 | 3.26 | |
| Llama-3.1-8B-CrabModel=Llama-3.1-8B-Crab2026.06 | 3.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.25 | 2.8 | 3.9 | 3.14 | 4.79 | 3.07 | |
| RLOOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.647 | 3.694 | 2.256 | 3.942 | 4.519 | 4.206 | 3.656 | 3.342 | 3.306 | 2.952 | 4.8 | 4.899 | 2.875 | 2.96 | — | — | — | — | — | — | |
| Llama-3.1-8B-RoleMRC2026.05 | 3.641 | 4.169 | 2.163 | 3.717 | 4.588 | 4.231 | 3.606 | 3.225 | 3.125 | 2.864 | 4.8 | 4.748 | 3.04 | 3.06 | — | — | — | — | — | — | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.06 | 3.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.64 | 2.95 | 3.94 | 3.12 | 4.87 | 3.03 | |
| Llama-3.1-8B-CoSERModel=Llama-3.1-8B-CoSER2026.06 | 3.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.15 | 2.92 | 3.92 | 3 | 4.88 | 3 | |
| Qwen2.5-14B-Instruct-AdaMARPModel=Qwen2.5-14B-Instruct-AdaMARP2026.06 | 3.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.04 | 3.02 | 3.83 | 3.04 | 4.8 | 3.17 | |
| Qwen2.5-7B-Instruct-CrabModel=Qwen2.5-7B-Instruct-Crab2026.06 | 3.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.16 | 2.78 | 3.85 | 3.12 | 4.8 | 3.07 | |
| PPOBackbone=Llama-3.2-3B-Instruct2026.05 | 3.517 | 3.888 | 2.75 | 4.108 | 4.506 | 4.306 | 3.231 | 3.292 | 2.594 | 2.619 | 4.8 | 5 | 2.22 | 2.405 | — | — | — | — | — | — | |
| Qwen2.5-7B-Instruct-CoSERModel=Qwen2.5-7B-Instruct-CoSER2026.06 | 3.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.02 | 2.94 | 3.67 | 2.86 | 4.69 | 2.95 | |
| ReMaxBackbone=Llama-3.2-3B-Instruct2026.05 | 3.482 | 4.063 | 2.363 | 4.058 | 4.531 | 4.338 | 3.281 | 3.358 | 2.594 | 2.575 | 4.868 | 5 | 2.035 | 2.2 | — | — | — | — | — | — | |
| CharacterGLM2026.05 | 3.461 | 3.76 | 2.18 | 3.97 | 4.03 | 3.8 | 3.26 | 2.89 | 2.94 | 2.64 | 4.53 | 4.51 | 3.16 | 3.32 | — | — | — | — | — | — | |
| NeekoBackbone=Llama-3.2-3B-Instruct2026.05 | 3.451 | 3.796 | 1.855 | 3.816 | 4.513 | 3.428 | 3.416 | 3.192 | 3.319 | 2.747 | 4.561 | 4.582 | 2.665 | 2.972 | — | — | — | — | — | — | |
| Peach-9B-8k-Roleplay2026.05 | 3.306 | 2.931 | 2.344 | 3.717 | 3.8 | 3.388 | 3.3 | 3.292 | 2.863 | 2.802 | 4.838 | 4.798 | 2.475 | 2.435 | — | — | — | — | — | — | |
| CharacterGLM-6B2026.05 | 3.213 | 3.245 | 2.1 | 3.543 | 3.365 | 3.41 | 3.07 | 3.1 | 2.61 | 2.5 | 4.48 | 4.8 | 2.84 | 2.7 | — | — | — | — | — | — | |
| REINFORCE++Backbone=Llama-3.2-3B-Instruct2026.05 | 3.045 | 2.794 | 2.231 | 3.392 | 3.625 | 2.825 | 2.588 | 3.158 | 2.519 | 2.481 | 4.838 | 4.924 | 1.98 | 2.23 | — | — | — | — | — | — |