General Reasoning on ZebraLogic (Accuracy (%))
96.3Accuracy (%)TeamTR (Heterogeneous)
Evaluation results
| Method | Links | |
|---|---|---|
| TeamTR (Heterogeneous)Size=1.7B+8B+14B2026.05 | 96.3 | |
| TeamTR (Homogeneous)Size=3×8B2026.05 | 94.7 | |
| Qwen3-Base SATSize=3×4B2026.04 | 93.1 | |
| Qwen3-A3BSize=30B2026.04 | 89.5 | |
| Qwen3-A3BSize=30B2026.05 | 89.5 | |
| OpenAI o3-mini (medium)Size=/2026.04 | 88.9 | |
| o4-miniSize=–2026.05 | 88.9 | |
| Qwen3 (thinking)Size=32B2026.04 | 88.8 | |
| Qwen3 (thinking)Size=32B2026.05 | 88.8 | |
| Qwen3-Base Debate + JudgeSize=3×8B2026.04 | 86.3 | |
| Debate + JudgeSize=3×8B2026.05 | 86.3 | |
| Qwen3-Base Role-playSize=3×8B2026.04 | 85.1 | |
| Role-playSize=3×8B2026.05 | 85.1 | |
| Qwen3-Base DebateSize=3×8B2026.04 | 84.1 | |
| DebateSize=3×8B2026.05 | 84.1 | |
| Llama 3.1-Base SATSize=3×8B2026.04 | 80.7 | |
| QwQSize=32B2026.04 | 76.8 | |
| QwQSize=32B2026.05 | 76.8 | |
| DeepSeek-R1 Distill LlamaSize=70B2026.04 | 71.3 | |
| DeepSeek-R1 DistillSize=70B2026.05 | 71.3 | |
| Qwen3-Base-DAPOSize=8B2026.04 | 36.7 | |
| DAPOSize=8B2026.05 | 36.7 | |
| Qwen3-Base-GRPOSize=8B2026.04 | 35.4 | |
| GRPOSize=8B2026.05 | 35.4 | |
| Qwen3-BaseSize=14B2026.04 | 33.1 | |
| Qwen3-BaseSize=32B2026.04 | 29.2 | |
| Qwen2.5-BaseSize=72B2026.04 | 26.6 | |
| GPT-4o-mini-2024-07-18Size=/2026.04 | 20.1 | |
| GPT-4o-miniSize=–2026.05 | 20.1 |