Role-playing on CharacterBench latest (full)
4.525Overall ScoreCRPO
Evaluation Results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CRPOBase Model=Qwen3-8B2026.05 | 4.525 | 2.581 | 4.308 | 4.994 | 4.781 | 4.469 | 3.717 | 3.819 | 3.354 | 4.925 | 4.659 | 3.3 | 3.13 | 4.043 | |
| OARBase Model=Qwen3-8B2026.05 | 4.45 | 2.369 | 4.092 | 4.944 | 4.588 | 4.125 | 3.333 | 3.475 | 3.21 | 4.913 | 4.849 | 3.04 | 3.075 | 3.882 | |
| CRPOBase Model=Qwen3-8B, Ablation=w/o Contrastive Anchor2026.05 | 4.444 | 2.488 | 4.192 | 4.994 | 4.569 | 4.219 | 3.417 | 3.663 | 3.317 | 4.888 | 4.823 | 3.215 | 3.025 | 3.942 | |
| GDPOBase Model=Qwen3-8B2026.05 | 4.425 | 2.4 | 4 | 4.775 | 4.55 | 4.075 | 3.367 | 3.594 | 3.147 | 4.875 | 4.785 | 3.1 | 3.09 | 3.86 | |
| CRPOBase Model=Qwen3-8B, Ablation=w/o Adaptive Exploitation2026.05 | 4.363 | 2.553 | 4.253 | 4.969 | 4.769 | 4.463 | 3.717 | 3.55 | 3.241 | 4.875 | 4.635 | 3.015 | 3.11 | 3.962 | |
| CRPOBase Model=Llama-3.2-3B-Instruct2026.05 | 4.356 | 2.113 | 4.175 | 4.988 | 4.619 | 4.038 | 3.483 | 3.581 | 3.166 | 4.888 | 4.873 | 2.75 | 2.785 | 3.832 | |
| GSPOBase Model=Qwen3-8B2026.05 | 4.344 | 2.481 | 4.125 | 4.575 | 4.55 | 4.169 | 3.608 | 3.684 | 3.172 | 4.75 | 4.659 | 3.255 | 3.255 | 3.894 | |
| GDPOBase Model=Llama-3.2-3B-Instruct2026.05 | 4.319 | 2.263 | 3.933 | 4.85 | 4.3 | 3.8 | 3.233 | 3.475 | 3.134 | 4.868 | 4.886 | 2.85 | 2.855 | 3.751 | |
| PPOBase Model=Qwen3-8B2026.05 | 4.306 | 2.244 | 4.2 | 4.844 | 4.569 | 4.169 | 3.583 | 3.575 | 3.091 | 4.85 | 4.722 | 3.335 | 3.225 | 3.901 | |
| Character-R1Base Model=Qwen3-8B2026.05 | 4.294 | 2.488 | 4.175 | 4.763 | 4.563 | 4.125 | 3.442 | 3.494 | 3.191 | 4.863 | 4.785 | 3.145 | 3.09 | 3.878 | |
| Character-R1Base Model=Llama-3.2-3B-Instruct2026.05 | 4.275 | 2.144 | 3.983 | 4.95 | 4.556 | 4.038 | 3.35 | 3.463 | 3.134 | 4.85 | 4.748 | 2.94 | 2.97 | 3.8 | |
| GSPOBase Model=Llama-3.2-3B-Instruct2026.05 | 4.238 | 2.156 | 4.15 | 4.838 | 4.45 | 3.806 | 3.208 | 3.444 | 3.041 | 4.825 | 4.811 | 2.975 | 2.94 | 3.76 | |
| OARBase Model=Llama-3.2-3B-Instruct2026.05 | 4.206 | 2.175 | 4.175 | 4.844 | 4.369 | 3.844 | 3.167 | 3.581 | 3.097 | 4.868 | 4.736 | 2.875 | 2.97 | 3.762 | |
| Qwen2.5-7B-RoleMRC2026.05 | 4.175 | 2.231 | 3.775 | 4.713 | 4.3 | 3.688 | 3.225 | 3.188 | 2.927 | 4.75 | 4.735 | 3 | 3.105 | 3.678 | |
| Llama-3.1-8B-RoleMRC2026.05 | 4.169 | 2.163 | 3.717 | 4.588 | 4.231 | 3.606 | 3.225 | 3.125 | 2.864 | 4.8 | 4.748 | 3.04 | 3.06 | 3.641 | |
| CRPOBase Model=Qwen3-8B, Ablation=w/o Dual-Stream Advantage2026.05 | 4.15 | 2.494 | 4.067 | 4.294 | 4.338 | 4.175 | 3.775 | 3.513 | 3.228 | 4.888 | 4.697 | 3.01 | 2.915 | 3.811 | |
| Crab2026.05 | 4.131 | 2.175 | 3.6 | 4.65 | 4.25 | 3.681 | 3.167 | 3.331 | 2.965 | 4.85 | 4.684 | 3.05 | 3.11 | 3.665 | |
| DAPOBase Model=Qwen3-8B2026.05 | 4.125 | 2.444 | 4.1 | 4.25 | 4.413 | 4.031 | 3.208 | 3.488 | 3.172 | 4.9 | 4.773 | 3.27 | 3.15 | 3.794 | |
| REINFORCE++Base Model=Qwen3-8B2026.05 | 4.1 | 2.419 | 4.092 | 4.7 | 4.344 | 3.806 | 3.217 | 3.213 | 2.783 | 4.9 | 4.899 | 3.5 | 3.211 | 3.783 | |
| Haruhi-Zero-7B2026.05 | 4.088 | 2.15 | 3.583 | 4.525 | 4.256 | 3.613 | 3.192 | 3.456 | 2.965 | 4.85 | 4.773 | 2.99 | 3.065 | 3.654 | |
| Qwen2.5-72B-Instruct2026.05 | 4.06 | 2.558 | 4.102 | 4.531 | 3.222 | 3.917 | 3.439 | 3.398 | 2.962 | 4.897 | 4.815 | 3.512 | 3.418 | 3.756 | |
| CoSER-Llama-3.1-8B2026.05 | 4.056 | 2.113 | 3.958 | 4.506 | 4.213 | 3.7 | 3.25 | 3.156 | 2.883 | 4.95 | 4.874 | 3.045 | 2.97 | 3.667 | |
| SFTBase Model=Qwen3-8B2026.05 | 4.025 | 2.219 | 3.925 | 4.731 | 4.331 | 3.65 | 3.208 | 3.25 | 2.795 | 4.8 | 4.71 | 3.03 | 3.075 | 3.673 | |
| Humanish-Roleplay2026.05 | 4.013 | 2.35 | 4 | 4.694 | 4.481 | 3.975 | 3.267 | 3.444 | 3.053 | 4.75 | 4.823 | 3.01 | 3.145 | 3.77 | |
| Claude-4-opus2026.05 | 3.997 | 2.435 | 4.398 | 4.508 | 4.362 | 3.877 | 3.751 | 3.587 | 3.176 | 4.936 | 4.735 | 3.151 | 3.351 | 3.866 | |
| NeekoBase Model=Qwen3-8B2026.05 | 3.991 | 2.202 | 3.83 | 4.709 | 4.132 | 3.725 | 3.439 | 3.28 | 2.83 | 4.853 | 4.767 | 3.112 | 3.182 | 3.696 | |
| Gemini-3-pro2026.05 | 3.946 | 2.905 | 4.099 | 4.167 | 4.133 | 3.671 | 3.529 | 3.319 | 2.973 | 4.82 | 4.633 | 3.613 | 3.423 | 3.787 | |
| Llama-3-70B-Instruct2026.05 | 3.94 | 2.59 | 3.95 | 4.39 | 3.96 | 3.33 | 3.35 | 3.06 | 2.89 | 4.71 | 4.74 | 3.4 | 3.51 | 3.678 | |
| GRPOBase Model=Qwen3-8B2026.05 | 3.913 | 2.456 | 4.125 | 4.194 | 4.413 | 4.094 | 3.617 | 3.444 | 3.097 | 4.863 | 4.861 | 3.405 | 3.295 | 3.829 | |
| RARBase Model=Qwen3-8B2026.05 | 3.892 | 2.537 | 4.129 | 4.485 | 4.32 | 4.342 | 3.646 | 3.135 | 2.781 | 4.873 | 4.693 | 3.15 | 3.144 | 3.779 | |
| PPOBase Model=Llama-3.2-3B-Instruct2026.05 | 3.888 | 2.75 | 4.108 | 4.506 | 4.306 | 3.231 | 3.292 | 2.594 | 2.619 | 4.8 | 5 | 2.22 | 2.405 | 3.517 | |
| Dr.GRPOBase Model=Qwen3-8B2026.05 | 3.869 | 2.574 | 4.092 | 3.681 | 4.406 | 4.131 | 3.6 | 3.625 | 3.153 | 4.825 | 4.76 | 3.05 | 3.1 | 3.759 | |
| RARBase Model=Llama-3.2-3B-Instruct2026.05 | 3.848 | 2.01 | 3.978 | 4.511 | 4.373 | 3.918 | 4.112 | 3.24 | 2.91 | 4.785 | 4.743 | 2.718 | 2.964 | 3.701 | |
| NeekoBase Model=Llama-3.2-3B-Instruct2026.05 | 3.796 | 1.855 | 3.816 | 4.513 | 3.428 | 3.416 | 3.192 | 3.319 | 2.747 | 4.561 | 4.582 | 2.665 | 2.972 | 3.451 | |
| GPT-4o2026.05 | 3.793 | 2.647 | 3.978 | 4.484 | 4.027 | 3.723 | 3.414 | 3.046 | 2.974 | 4.763 | 4.771 | 3.261 | 3.51 | 3.722 | |
| ReMaxBase Model=Qwen3-8B2026.05 | 3.781 | 2.425 | 3.792 | 4.45 | 4.206 | 3.638 | 3.583 | 2.963 | 2.783 | 4.875 | 4.76 | 3.475 | 3.275 | 3.693 | |
| CharacterGLM2026.05 | 3.76 | 2.18 | 3.97 | 4.03 | 3.8 | 3.26 | 2.89 | 2.94 | 2.64 | 4.53 | 4.51 | 3.16 | 3.32 | 3.461 | |
| Baichuan-NPC2026.05 | 3.672 | 2.134 | 4.132 | 4.254 | 4.216 | 4.022 | 3.366 | 3.001 | 3.179 | 4.83 | 4.897 | 2.975 | 3.297 | 3.69 | |
| SFTBase Model=Llama-3.2-3B-Instruct2026.05 | 3.606 | 2.063 | 3.892 | 4.55 | 4.169 | 3.894 | 3.5 | 3.219 | 3.003 | 4.888 | 4.823 | 3.35 | 3.31 | 3.713 | |
| Qwen3-8BBase Model=Qwen3-8B2026.05 | 3.594 | 2.4 | 3.758 | 4.556 | 4.188 | 3.819 | 3.692 | 3.094 | 2.808 | 4.8 | 4.684 | 3.445 | 3.27 | 3.701 | |
| Llama-3.2-3B-InstructBase Model=Llama-3.2-3B-Instruct2026.05 | 3.588 | 2.088 | 3.892 | 4.606 | 4.3 | 4.025 | 3.3 | 3.306 | 3.053 | 4.84 | 4.649 | 2.89 | 3.32 | 3.681 | |
| RLOOBase Model=Qwen3-8B2026.05 | 3.556 | 2.425 | 3.792 | 4.519 | 4.263 | 3.931 | 3.717 | 3.119 | 2.852 | 4.788 | 4.773 | 3.47 | 3.295 | 3.731 | |
| GLM-42026.05 | 3.524 | 2.373 | 3.701 | 4.38 | 4.103 | 3.728 | 3.487 | 3.13 | 2.987 | 4.826 | 4.876 | 3.208 | 3.5 | 3.679 | |
| GPT-3.5-turbo2026.05 | 3.49 | 2.451 | 3.692 | 4.345 | 4.155 | 3.635 | 3.56 | 3.09 | 2.838 | 4.735 | 4.761 | 3.619 | 3.758 | 3.702 | |
| CharacterGLM-6B2026.05 | 3.245 | 2.1 | 3.543 | 3.365 | 3.41 | 3.07 | 3.1 | 2.61 | 2.5 | 4.48 | 4.8 | 2.84 | 2.7 | 3.213 | |
| Peach-9B-8k-Roleplay2026.05 | 2.931 | 2.344 | 3.717 | 3.8 | 3.388 | 3.3 | 3.292 | 2.863 | 2.802 | 4.838 | 4.798 | 2.475 | 2.435 | 3.306 | |
| Peach-2.0-9B-8k-Roleplay2026.05 | 2.819 | 2.4 | 3.425 | 4.4 | 4.263 | 4.088 | 3.867 | 3.35 | 3.028 | 4.475 | 4.47 | 3.485 | 3.68 | 3.673 |