Role-play dialogue comprehension on SocialBench
96.6Role KnowledgeQwen3-8B + CRPO
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-8B + CRPOBase Model=Qwen3-8B2026.05 | 96.6 | 92.2 | 48.8 | 35 | 79 | 97 | 92.7 | 95.5 | 85.2 | 80.2 | |
| CRPOBackbone=Qwen3-8B2026.05 | 96.6 | 92.2 | 48.8 | 35 | 79 | 97 | 92.7 | 95.5 | 85.2 | 80.2 | |
| Qwen3-8B + GDPOBase Model=Qwen3-8B2026.05 | 95.1 | 89.6 | 45.2 | 28 | 72 | 94.3 | 91.3 | 94.6 | 82 | 76.9 | |
| GDPOBackbone=Qwen3-8B2026.05 | 95.1 | 89.6 | 45.2 | 28 | 72 | 94.3 | 91.3 | 94.6 | 82 | 76.9 | |
| Qwen3-8B + Character-R1Base Model=Qwen3-8B2026.05 | 94.9 | 88.4 | 44.9 | 28 | 76 | 94.1 | 91.9 | 94.2 | 85.4 | 77.5 | |
| Character-R1Backbone=Qwen3-8B2026.05 | 94.9 | 88.4 | 44.9 | 28 | 76 | 94.1 | 91.9 | 94.2 | 85.4 | 77.5 | |
| ReMaxBackbone=Qwen3-8B2026.05 | 94.9 | 89.6 | 45.2 | 23 | 85 | 91.4 | 91.9 | 95.5 | 83.1 | 77.7 | |
| Dr.GRPOBackbone=Qwen3-8B2026.05 | 94.6 | 88.1 | 51.1 | 23 | 87 | 93.6 | 92.7 | 93.4 | 82 | 78.4 | |
| RLOOBackbone=Qwen3-8B2026.05 | 94.3 | 89.5 | 44.7 | 28 | 81 | 91 | 91 | 95.5 | 88.1 | 78.1 | |
| Qwen3-8B + PPOBase Model=Qwen3-8B2026.05 | 94.2 | 87.6 | 48.9 | 28 | 86 | 91.5 | 91.3 | 93.8 | 82.6 | 78.2 | |
| Qwen3-8B + GSPOBase Model=Qwen3-8B2026.05 | 94.2 | 90.3 | 46.3 | 28 | 68 | 93.4 | 91.6 | 94.6 | 83.1 | 76.6 | |
| PPOBackbone=Qwen3-8B2026.05 | 94.2 | 87.6 | 48.9 | 28 | 86 | 91.5 | 91.3 | 93.8 | 82.6 | 78.2 | |
| GRPOBackbone=Qwen3-8B2026.05 | 94.2 | 89.2 | 45.5 | 28 | 77 | 92.6 | 92.7 | 96.3 | 83.1 | 77.6 | |
| GSPOBackbone=Qwen3-8B2026.05 | 94.2 | 90.3 | 46.3 | 28 | 68 | 93.4 | 91.6 | 94.6 | 83.1 | 76.6 | |
| Qwen3-8BBase Model=Qwen3-8B2026.05 | 93.8 | 88.8 | 45.7 | 23 | 74 | 91.8 | 92.6 | 95.5 | 87 | 76.9 | |
| Qwen3-8BBackbone=Qwen3-8B2026.05 | 93.8 | 88.8 | 45.7 | 23 | 74 | 91.8 | 92.6 | 95.5 | 87 | 76.9 | |
| Qwen3-8B + OARBase Model=Qwen3-8B2026.05 | 93.2 | 88.8 | 52.7 | 28 | 81 | 94.1 | 92.2 | 93.8 | 84.8 | 78.7 | |
| OARBackbone=Qwen3-8B2026.05 | 93.2 | 88.8 | 52.7 | 28 | 81 | 94.1 | 92.2 | 93.8 | 84.8 | 78.7 | |
| Qwen3-8B + DAPOBase Model=Qwen3-8B2026.05 | 93.1 | 87 | 46.2 | 25.6 | 77.2 | 88.1 | 91.6 | 93.9 | 83.1 | 76.2 | |
| DAPOBackbone=Qwen3-8B2026.05 | 93.1 | 87 | 46.2 | 25.6 | 77.2 | 88.1 | 91.6 | 93.9 | 83.1 | 76.2 | |
| REINFORCE++Backbone=Qwen3-8B2026.05 | 92.6 | 83.7 | 47.9 | 23 | 84 | 84.7 | 91.3 | 92.1 | 84.8 | 76 | |
| Qwen3-8B + NeekoBase Model=Qwen3-8B2026.05 | 92.4 | 80.4 | 43.5 | 26.7 | 68.8 | 78.5 | 86.6 | 92.5 | 77.5 | 71.9 | |
| NeekoBackbone=Qwen3-8B2026.05 | 92.4 | 80.4 | 43.5 | 26.7 | 68.8 | 78.5 | 86.6 | 92.5 | 77.5 | 71.9 | |
| Qwen3-8B + SFTBase Model=Qwen3-8B, Training=SFT2026.05 | 91.3 | 81.4 | 48.7 | 23 | 81 | 85.5 | 88.1 | 89.6 | 78.8 | 74.1 | |
| SFTBackbone=Qwen3-8B2026.05 | 91.3 | 81.4 | 48.7 | 23 | 81 | 85.5 | 88.1 | 89.6 | 78.8 | 74.1 | |
| Qwen3-8B + RARBase Model=Qwen3-8B2026.05 | 91.1 | 83.2 | 48.5 | 25 | 80.7 | 92.9 | 88.9 | 91.5 | 81.7 | 76 | |
| RARBackbone=Qwen3-8B2026.05 | 91.1 | 83.2 | 48.5 | 25 | 80.7 | 92.9 | 88.9 | 91.5 | 81.7 | 76 | |
| Humanish-Roleplay2026.05 | 89.9 | 80.8 | 34.3 | 29.2 | 92 | 84.2 | 84.3 | 89.9 | 74.2 | 73.2 | |
| Humanish-Roleplay2026.05 | 89.9 | 80.8 | 34.3 | 29.2 | 92 | 84.2 | 84.3 | 89.9 | 74.2 | 73.2 | |
| Peach-2.0-9B-8k-Roleplay2026.05 | 89.3 | 72.8 | 46.7 | 26.3 | 87 | 48.4 | 83.7 | 90.3 | 85.7 | 70 | |
| Peach-2.0-9B-8k-Roleplay2026.05 | 89.3 | 72.8 | 46.7 | 26.3 | 87 | 48.4 | 83.7 | 90.3 | 85.7 | 70 | |
| Qwen2.5-7B-RoleMRCBase Model=Qwen2.5-7B2026.05 | 86.8 | 75.7 | 43.4 | 68.3 | 84 | 45.9 | 81.4 | 83.1 | 76.4 | 71.7 | |
| Qwen2.5-7B-RoleMRC2026.05 | 86.8 | 75.7 | 43.4 | 68.3 | 84 | 45.9 | 81.4 | 83.1 | 76.4 | 71.7 | |
| REINFORCE++Backbone=Llama-3.2-3B-Instruct2026.05 | 85.5 | 71 | 41.8 | 31.7 | 71 | 60.5 | 79.8 | 85.2 | 79.1 | 67.3 | |
| CRPOBackbone=Llama-3.2-3B-Instruct2026.05 | 85.5 | 70.4 | 42.6 | 42.9 | 80 | 69 | 76.9 | 84 | 77.5 | 69.9 | |
| DAPOBackbone=Llama-3.2-3B-Instruct2026.05 | 85.2 | 70.4 | 40.3 | 28.6 | 70 | 53.4 | 77.5 | 85.1 | 76.5 | 65.2 | |
| GSPOBackbone=Llama-3.2-3B-Instruct2026.05 | 85.2 | 70.1 | 40.2 | 23.3 | 75 | 69 | 78.5 | 86.5 | 76.9 | 67.2 | |
| OARBackbone=Llama-3.2-3B-Instruct2026.05 | 85.1 | 70.7 | 40.2 | 27.9 | 73 | 69 | 79.2 | 84.8 | 78.6 | 67.6 | |
| Character-R1Backbone=Llama-3.2-3B-Instruct2026.05 | 84.7 | 72.7 | 40.5 | 26.7 | 73 | 56.4 | 77.6 | 81.6 | 67.6 | 64.5 | |
| GDPOBackbone=Llama-3.2-3B-Instruct2026.05 | 84.7 | 70.5 | 42.1 | 30 | 75 | 64.5 | 73.1 | 84 | 73.6 | 66.4 | |
| Dr.GRPOBackbone=Llama-3.2-3B-Instruct2026.05 | 84.1 | 72.2 | 40.9 | 18 | 73 | 55 | 82 | 85.3 | 73.8 | 64.9 | |
| CoSER-Llama-3.1-8BBase Model=Llama-3.1-8B2026.05 | 82.2 | 62.2 | 39.4 | 45.8 | 75 | 69.8 | 50.6 | 73.4 | 15.9 | 57.1 | |
| CoSER-Llama-3.1-8B2026.05 | 82.2 | 62.2 | 39.4 | 45.8 | 75 | 69.8 | 50.6 | 73.4 | 15.9 | 57.1 | |
| Llama-3.1-8B-RoleMRCBase Model=Llama-3.1-8B2026.05 | 81.6 | 74.9 | 42.4 | 33.8 | 65 | 65.7 | 75.6 | 81.9 | 65.4 | 65.1 | |
| Llama-3.1-8B-RoleMRC2026.05 | 81.6 | 74.9 | 42.4 | 33.8 | 65 | 65.7 | 75.6 | 81.9 | 65.4 | 65.1 | |
| GRPOBackbone=Llama-3.2-3B-Instruct2026.05 | 79.6 | 71.8 | 38.6 | 21 | 69 | 62.2 | 75.8 | 82.9 | 71.3 | 63.6 | |
| PPOBackbone=Llama-3.2-3B-Instruct2026.05 | 79.1 | 73.5 | 38.3 | 33.3 | 70 | 51.6 | 81.1 | 83.5 | 63.7 | 63.8 | |
| RARBackbone=Llama-3.2-3B-Instruct2026.05 | 78.1 | 70.5 | 41.5 | 29.6 | 73.5 | 51.2 | 73.7 | 79.7 | 63.5 | 62.4 | |
| ReMaxBackbone=Llama-3.2-3B-Instruct2026.05 | 78 | 70.2 | 41.6 | 21.7 | 71 | 44.7 | 71.5 | 81 | 63.7 | 60.4 | |
| Crab2026.05 | 77.9 | 64.1 | 38.9 | 52.5 | 72 | 59.1 | 69.9 | 82.3 | 46.7 | 62.6 | |
| Crab2026.05 | 77.9 | 64.1 | 38.9 | 52.5 | 72 | 59.1 | 69.9 | 82.3 | 46.7 | 62.6 | |
| RLOOBackbone=Llama-3.2-3B-Instruct2026.05 | 77.6 | 70.2 | 42.4 | 30.4 | 74 | 54.2 | 76.6 | 82.7 | 72 | 64.5 | |
| NeekoBackbone=Llama-3.2-3B-Instruct2026.05 | 77.1 | 65 | 39.8 | 29.6 | 57.6 | 44.6 | 72.8 | 81.6 | 60 | 58.7 | |
| CharacterGLM-6B2026.05 | 74.7 | 79.4 | 26.2 | 41.2 | 81.1 | 68.2 | 84.4 | 70.4 | 36.3 | 62.5 | |
| SFTBackbone=Llama-3.2-3B-Instruct2026.05 | 74.7 | 73.5 | 40 | 30.4 | 73 | 49.4 | 76.9 | 83.5 | 66.2 | 63.1 | |
| CharacterGLM-6B2026.05 | 74.7 | 79.4 | 26.2 | 41.2 | 81.1 | 68.2 | 84.4 | 70.4 | 36.3 | 62.5 | |
| Haruhi-Zero-7B2026.05 | 74.2 | 71.5 | 29 | 40.8 | 57 | 43.1 | 57.7 | 72.2 | 29.1 | 52.7 | |
| Haruhi-Zero-7B2026.05 | 74.2 | 71.5 | 29 | 40.8 | 57 | 43.1 | 57.7 | 72.2 | 29.1 | 52.7 | |
| Llama-3.2-3B-InstructBackbone=Llama-3.2-3B-Instruct2026.05 | 73.5 | 72.5 | 40.4 | 35.4 | 73 | 50.4 | 76.9 | 80.5 | 66.9 | 63.3 | |
| Peach-9B-8k-Roleplay2026.05 | 58 | 25.3 | 40.8 | 31.3 | 67 | 46.4 | 60.9 | 67.1 | 48.4 | 49.4 | |
| Peach-9B-8k-Roleplay2026.05 | 58 | 25.3 | 40.8 | 31.3 | 67 | 46.4 | 60.9 | 67.1 | 48.4 | 49.4 |