Role-Playing on RPGBench Dialogue Shift (Generalization)
-0.956Turn CompositionRFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RFTBackbone=Qwen3-4B2025.12 | -0.956 | -0.583 | |
| DPOBackbone=Meta-Llama3-8B2025.12 | -0.565 | -0.387 | |
| ICLBackbone=Meta-Llama3-8B2025.12 | -0.415 | -0.014 | |
| ICLBackbone=Qwen3-4B2025.12 | -0.362 | -0.018 | |
| DPOBackbone=Qwen3-4B2025.12 | -0.292 | -0.111 | |
| RFTBackbone=Meta-Llama3-8B2025.12 | -0.241 | -0.127 | |
| ThinkingRLBackbone=Meta-Llama3-8B2025.12 | -0.218 | -0.059 | |
| ThinkingRLBackbone=Qwen3-4B2025.12 | -0.175 | -0.007 | |
| ThinkingSFTBackbone=Qwen3-4B2025.12 | -0.145 | 0.078 | |
| ThinkingSFTBackbone=Meta-Llama3-8B2025.12 | -0.105 | 0.116 | |
| RLBackbone=Meta-Llama3-8B2025.12 | -0.101 | 0.164 | |
| DABackbone=Qwen3-4B2025.12 | -0.098 | 0.137 | |
| SFTBackbone=Meta-Llama3-8B2025.12 | -0.081 | 0.132 | |
| DABackbone=Meta-Llama3-8B2025.12 | -0.08 | 0.146 | |
| CoRLBackbone=Qwen3-4B2025.12 | -0.066 | 0.153 | |
| RLBackbone=Qwen3-4B2025.12 | -0.057 | 0.149 | |
| SFTBackbone=Qwen3-4B2025.12 | -0.049 | 0.143 | |
| CoRLBackbone=Meta-Llama3-8B2025.12 | -0.044 | 0.181 |