Logical Reasoning on Zebra
36.5Pass@1 AccuracySEC
Evaluation results
| Method | Links | |
|---|---|---|
| SECBackbone=Llama3.2-3B2026.06 | 36.5 | |
| TACBackbone=Llama3.2-3B2026.06 | 35.8 | |
| TACBackbone=Qwen3-1.7B2026.06 | 34.7 | |
| M2OBackbone=Llama3.2-3B2026.06 | 30.9 | |
| RandomBackbone=Llama3.2-3B2026.06 | 30.5 | |
| M2OBackbone=Qwen3-1.7B2026.06 | 28.5 | |
| SECBackbone=Qwen3-1.7B2026.06 | 26.7 | |
| RandomBackbone=Qwen3-1.7B2026.06 | 24.9 | |
| BaseBackbone=Llama3.2-3B2026.06 | 1.4 | |
| BaseBackbone=Qwen3-1.7B2026.06 | 1.1 |