Reasoning on BBH (T, U, F Metrics)
71.5T Score (BBH)Binary
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BinaryBackbone=Qwen2.5-7B-Instruct2026.04 | 71.5 | 0.1 | 28.4 | |
| KARLBackbone=Qwen2.5-7B-Instruct2026.04 | 67.8 | 7.4 | 24.8 | |
| PromptingBackbone=Qwen2.5-7B-Instruct2026.04 | 67.5 | 5.8 | 26.7 | |
| PromptingBackbone=Llama-3.1-8B-Instruct2026.04 | 64.3 | 2.1 | 33.6 | |
| BinaryBackbone=Llama-3.1-8B-Instruct2026.04 | 63.2 | 0 | 36.8 | |
| KARLBackbone=Llama-3.1-8B-Instruct2026.04 | 62.5 | 11.3 | 26.2 | |
| TernaryBackbone=Qwen2.5-7B-Instruct2026.04 | 61.5 | 15.6 | 22.9 | |
| TernaryBackbone=Llama-3.1-8B-Instruct2026.04 | 53.4 | 28.1 | 18.5 |