Reasoning on CALI OOD (test)
77.95AccuracyMultiRole-R1
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MultiRole-R1Backbone=Qwen3-8B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 77.95 | 83.84 | |
| SelfConsis SFT+GRPOBackbone=Qwen3-8B2025.07 | 77.83 | 80.36 | |
| MultiRole-R1Backbone=R1-Distill-Qwen-14B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 76.5 | 89.08 | |
| SelfConsis SFT+DPOBackbone=Qwen3-8B2025.07 | 76.1 | 69.58 | |
| SelfConsis SFTBackbone=R1-Distill-Qwen-14B2025.07 | 76.08 | 73.65 | |
| SelfConsis SFTBackbone=Qwen3-8B2025.07 | 75.96 | 72.78 | |
| More thinkBackbone=R1-Distill-Qwen-14B2025.07 | 75.9 | 76.81 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Qwen-14B2025.07 | 75.82 | 71.71 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Qwen-14B2025.07 | 75.65 | 84.92 | |
| More thinkBackbone=Qwen3-8B2025.07 | 75.1 | 68.44 | |
| Zero-shot CoTBackbone=R1-Distill-Qwen-14B2025.07 | 75.05 | 71.83 | |
| Zero-shot CoTBackbone=Qwen3-8B2025.07 | 73.4 | 57.43 | |
| MultiRole-R1Backbone=R1-Distill-Llama-8B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 71.48 | 90.55 | |
| Self-RefineBackbone=R1-Distill-Qwen-14B2025.07 | 71.28 | 78.22 | |
| MultiRole-R1Backbone=R1-Distill-Qwen-7B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 70.85 | 83.31 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Qwen-7B2025.07 | 70.83 | 82.15 | |
| Role-PlayBackbone=Qwen3-8B2025.07 | 70.26 | 50.49 | |
| SelfConsis SFTBackbone=R1-Distill-Llama-8B2025.07 | 70.05 | 79.77 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Llama-8B2025.07 | 69.81 | 76.62 | |
| Self-RefineBackbone=Qwen3-8B2025.07 | 69.4 | 48.16 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Llama-8B2025.07 | 69.26 | 83.37 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Qwen-7B2025.07 | 68.19 | 64.09 | |
| Role-PlayBackbone=R1-Distill-Qwen-14B2025.07 | 67.41 | 70.74 | |
| SelfConsis SFTBackbone=R1-Distill-Qwen-7B2025.07 | 67.35 | 78.94 | |
| More thinkBackbone=R1-Distill-Llama-8B2025.07 | 64.41 | 80.3 | |
| Role-PlayBackbone=R1-Distill-Llama-8B2025.07 | 62.7 | 77.32 | |
| Self-RefineBackbone=R1-Distill-Llama-8B2025.07 | 61.95 | 78.87 | |
| Zero-shot CoTBackbone=R1-Distill-Llama-8B2025.07 | 60.84 | 73.98 | |
| More thinkBackbone=R1-Distill-Qwen-7B2025.07 | 60.45 | 78.82 | |
| Role-PlayBackbone=R1-Distill-Qwen-7B2025.07 | 52.69 | 67.43 | |
| Self-RefineBackbone=R1-Distill-Qwen-7B2025.07 | 50.76 | 66.09 | |
| Zero-shot CoTBackbone=R1-Distill-Qwen-7B2025.07 | 50.3 | 52.22 |