Reasoning on GLOQA (test)
53.98AccuracyMultiRole-R1
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MultiRole-R1Backbone=R1-Distill-Qwen-14B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 53.98 | 91.32 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Qwen-14B2025.07 | 51.73 | 90.33 | |
| MultiRole-R1Backbone=Qwen3-8B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 51.72 | 89.88 | |
| SelfConsis SFT+GRPOBackbone=Qwen3-8B2025.07 | 51.37 | 87.74 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Qwen-14B2025.07 | 51.33 | 65.03 | |
| SelfConsis SFTBackbone=R1-Distill-Qwen-14B2025.07 | 50.98 | 81.04 | |
| SelfConsis SFT+DPOBackbone=Qwen3-8B2025.07 | 50.68 | 74.14 | |
| SelfConsis SFTBackbone=Qwen3-8B2025.07 | 50.32 | 77.07 | |
| Role-PlayBackbone=R1-Distill-Qwen-14B2025.07 | 49.9 | 75.73 | |
| MultiRole-R1Backbone=R1-Distill-Qwen-7B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 49.1 | 89.67 | |
| MultiRole-R1Backbone=R1-Distill-Llama-8B, Method Strategy=SelfConsis SFT+GRPO(RS)2025.07 | 49.06 | 91.78 | |
| Self-RefineBackbone=R1-Distill-Qwen-14B2025.07 | 49.04 | 69.4 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Llama-8B2025.07 | 48.89 | 80.97 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Llama-8B2025.07 | 48.55 | 89.36 | |
| SelfConsis SFTBackbone=R1-Distill-Llama-8B2025.07 | 48.17 | 87.26 | |
| Role-PlayBackbone=Qwen3-8B2025.07 | 47.57 | 54.89 | |
| SelfConsis SFT+GRPOBackbone=R1-Distill-Qwen-7B2025.07 | 47.22 | 87.46 | |
| Self-RefineBackbone=Qwen3-8B2025.07 | 45.25 | 85 | |
| Role-PlayBackbone=R1-Distill-Llama-8B2025.07 | 44.87 | 83.02 | |
| SelfConsis SFT+DPOBackbone=R1-Distill-Qwen-7B2025.07 | 44.2 | 61.17 | |
| More thinkBackbone=R1-Distill-Llama-8B2025.07 | 44.04 | 87.19 | |
| More thinkBackbone=Qwen3-8B2025.07 | 43.39 | 78.98 | |
| Self-RefineBackbone=R1-Distill-Llama-8B2025.07 | 43.19 | 81.11 | |
| Self-RefineBackbone=R1-Distill-Qwen-7B2025.07 | 43.13 | 59.88 | |
| SelfConsis SFTBackbone=R1-Distill-Qwen-7B2025.07 | 43.13 | 85.58 | |
| Zero-shot CoTBackbone=Qwen3-8B2025.07 | 42.13 | 60.99 | |
| Role-PlayBackbone=R1-Distill-Qwen-7B2025.07 | 41.67 | 77.75 | |
| More thinkBackbone=R1-Distill-Qwen-14B2025.07 | 41.6 | 84.04 | |
| Zero-shot CoTBackbone=R1-Distill-Llama-8B2025.07 | 38.41 | 87.07 | |
| Zero-shot CoTBackbone=R1-Distill-Qwen-14B2025.07 | 36.82 | 79.18 | |
| More thinkBackbone=R1-Distill-Qwen-7B2025.07 | 36.42 | 86.9 | |
| Zero-shot CoTBackbone=R1-Distill-Qwen-7B2025.07 | 32.62 | 65.88 |