Reasoning on BBQ (test)
97.5Accuracy (Reasoning BBQ)MultiRole-R1
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MultiRole-R1Backbone=R1-Distill-Qwen-14B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 97.5 | 90.17 | |
| MultiRole-R1Backbone=Qwen3-8B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 96.98 | 88.15 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Qwen-14B2025.07 | 95.98 | 86.88 | |
| SelfConsis SFT+GRPOBackbone=Qwen3-8B2025.07 | 95.91 | 85.47 | |
| MultiRole-R1Backbone=R1-Distill-Llama-8B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 95.55 | 89.58 | |
| More thinkBackbone=Qwen3-8B2025.07 | 95.18 | 74.2 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Qwen-14B2025.07 | 94.98 | 67.21 | |
| More thinkBackbone=R1-Distill-Qwen-14B2025.07 | 94.57 | 80.67 | |
| MultiRole-R1Backbone=R1-Distill-Qwen-7B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 94.5 | 86.25 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Llama-8B2025.07 | 94.47 | 85.75 | |
| SelfConsis SFTBackbone=R1-Distill-Qwen-14B2025.07 | 94.4 | 75.06 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Qwen-7B2025.07 | 94.3 | 85.52 | |
| SelfConsis SFT+DPOBackbone=Qwen3-8B2025.07 | 94.21 | 74.83 | |
| SelfConsis SFTBackbone=Qwen3-8B2025.07 | 94.05 | 74.02 | |
| Zero-shot CoTBackbone=Qwen3-8B2025.07 | 91.71 | 70.1 | |
| Role-PlayBackbone=R1-Distill-Qwen-14B2025.07 | 91.18 | 81.87 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Llama-8B2025.07 | 90.52 | 80.43 | |
| Self-RefineBackbone=R1-Distill-Qwen-14B2025.07 | 90.42 | 80.13 | |
| Role-PlayBackbone=Qwen3-8B2025.07 | 89.77 | 41.58 | |
| SelfConsis SFTBackbone=R1-Distill-Llama-8B2025.07 | 89.69 | 82.64 | |
| Self-RefineBackbone=Qwen3-8B2025.07 | 88.93 | 53.07 | |
| More thinkBackbone=R1-Distill-Llama-8B2025.07 | 88.2 | 84.11 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Qwen-7B2025.07 | 86.41 | 60.43 | |
| SelfConsis SFTBackbone=R1-Distill-Qwen-7B2025.07 | 85.88 | 81.67 | |
| Zero-shot CoTBackbone=R1-Distill-Qwen-14B2025.07 | 85.01 | 68.06 | |
| Zero-shot CoTBackbone=R1-Distill-Llama-8B2025.07 | 80.89 | 79.92 | |
| More thinkBackbone=R1-Distill-Qwen-7B2025.07 | 80.76 | 80.44 | |
| Role-PlayBackbone=R1-Distill-Llama-8B2025.07 | 74.4 | 80.91 | |
| Self-RefineBackbone=R1-Distill-Llama-8B2025.07 | 74.2 | 75.85 | |
| Self-RefineBackbone=R1-Distill-Qwen-7B2025.07 | 74.08 | 73.13 | |
| Role-PlayBackbone=R1-Distill-Qwen-7B2025.07 | 73.61 | 74.68 | |
| Zero-shot CoTBackbone=R1-Distill-Qwen-7B2025.07 | 62.45 | 56.02 |