Role-Playing on RPGBench User Shift Generalization
-0.016RP Score (German)CoRL
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CoRLBackbone=Qwen3-4B2025.12 | -0.016 | -0.033 | -0.113 | -0.036 | -0.002 | |
| CoRLBackbone=Meta-Llama3-8B2025.12 | -0.017 | -0.077 | -0.315 | -0.162 | -0.12 | |
| DABackbone=Qwen3-4B2025.12 | -0.019 | -0.058 | -0.133 | -0.083 | -0.047 | |
| RLBackbone=Qwen3-4B2025.12 | -0.036 | -0.055 | -0.104 | -0.076 | -0.045 | |
| ThinkingSFTBackbone=Meta-Llama3-8B2025.12 | -0.04 | -0.118 | -0.345 | -0.201 | -0.144 | |
| SFTBackbone=Qwen3-4B2025.12 | -0.047 | -0.051 | -0.118 | -0.096 | -0.07 | |
| RLBackbone=Meta-Llama3-8B2025.12 | -0.048 | -0.115 | -0.364 | -0.2 | -0.159 | |
| DABackbone=Meta-Llama3-8B2025.12 | -0.049 | -0.108 | -0.335 | -0.198 | -0.144 | |
| SFTBackbone=Meta-Llama3-8B2025.12 | -0.065 | -0.092 | -0.373 | -0.206 | -0.145 | |
| ThinkingSFTBackbone=Qwen3-4B2025.12 | -0.085 | -0.116 | -0.32 | -0.087 | -0.066 | |
| ThinkingRLBackbone=Meta-Llama3-8B2025.12 | -0.113 | -0.219 | -0.442 | -0.243 | -0.186 | |
| DPOBackbone=Qwen3-4B2025.12 | -0.125 | -0.218 | -0.387 | -0.159 | -0.091 | |
| RFTBackbone=Meta-Llama3-8B2025.12 | -0.13 | -0.204 | -0.492 | -0.296 | -0.259 | |
| ThinkingRLBackbone=Qwen3-4B2025.12 | -0.206 | -0.185 | -0.269 | -0.156 | -0.292 | |
| ICLBackbone=Qwen3-4B2025.12 | -0.221 | -0.245 | -0.43 | -0.295 | -0.225 | |
| DPOBackbone=Meta-Llama3-8B2025.12 | -0.312 | -0.371 | -0.678 | -0.421 | -0.273 | |
| ICLBackbone=Meta-Llama3-8B2025.12 | -0.371 | -0.463 | -0.881 | -0.489 | -0.33 | |
| RFTBackbone=Qwen3-4B2025.12 | -0.491 | -0.671 | -0.853 | -0.628 | -0.427 |