Reasoning on BBEH mini
14.8Pass@1SUPERNOVA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SUPERNOVAModel Size=4B2026.04 | 14.8 | 31.7 | |
| SUPERNOVAModel Size=1.7B2026.04 | 12.4 | 25.7 | |
| Qwen3Model Size=4B2026.04 | 12 | 23.7 | |
| Qwen3Model Size=8B2026.04 | 11.9 | 21.7 | |
| General-ReasonerModel Size=4B2026.04 | 11.5 | 30 | |
| SUPERNOVAModel Size=0.6B2026.04 | 9.8 | 24.6 | |
| Qwen3Model Size=1.7B2026.04 | 7.6 | 18 | |
| Qwen3.5Model Size=2B2026.04 | 7.5 | 22.4 | |
| Qwen3Model Size=0.8B2026.04 | 6.6 | 22.2 | |
| Olmo-3-ThinkModel Size=7B2026.04 | 6.4 | 13 | |
| Qwen3Model Size=0.6B2026.04 | 5.7 | 15.4 | |
| OpenReasoning-NemotronModel Size=7B2026.04 | 3.5 | 8.5 | |
| Openthinker3Model Size=7B2026.04 | 3.4 | 7.6 |