Reasoning on BigBenchHard
100Accuracy (BigBenchHard)Ll4-Sct
Evaluation Results
| Method | Links | |
|---|---|---|
| Ll4-Sct2026.05 | 100 | |
| Gm2.5fl2026.05 | 99.19 | |
| Gemma 3 27BParameters=27B2026.02 | 82.4 | |
| General-Reasoner-14BModel Category=Frontier Models2026.04 | 78.2 | |
| TTRLBase Model=Qwen3-8B-Base2026.04 | 74.9 | |
| TEMPOBase Model=Qwen3-8B-Base2026.04 | 74.2 | |
| DictaLM 3.0 24B-ThinkParameters=24B, Variant=Thinking2026.02 | 73 | |
| Mistral Small 3.12026.02 | 71.91 | |
| DictaLM 3.0 12B-InstParameters=12B, Variant=Instruct2026.02 | 71.38 | |
| Gemma 3 12BParameters=12B2026.02 | 70.1 | |
| Zero-RL (PPO)Base Model=Qwen3-8B-Base2026.04 | 69.9 | |
| CT-KVPretrained Model=Qwen3-32B2025.07 | 69.1 | |
| TEMPOBase Model=OLMO3-7B-Base2026.04 | 68.2 | |
| EMPOBase Model=Qwen3-8B-Base2026.04 | 66.7 | |
| CT-KVPretrained Model=DeepSeek-32B2025.07 | 66.7 | |
| General-Reasoner-7BModel Category=Frontier Models2026.04 | 65.6 | |
| CT-KVPretrained Model=Qwen3-14B2025.07 | 64.2 | |
| ICLPretrained Model=DeepSeek-32B2025.07 | 63.5 | |
| ICLPretrained Model=Qwen3-32B2025.07 | 63.1 | |
| MiMo-Zero-RL-7BModel Category=Frontier Models2026.04 | 61.4 | |
| ICLPretrained Model=Qwen3-14B2025.07 | 60.8 | |
| CT-KVPretrained Model=DeepSeek-14B2025.07 | 58.4 | |
| CT-KVPretrained Model=Mistral-12B2025.07 | 57.4 | |
| Olmo-3-7B-RL-Zero-GeneralModel Category=Frontier Models2026.04 | 56.5 | |
| Prefix TuningPretrained Model=Qwen3-32B, m=322025.07 | 56.4 | |
| ICLPretrained Model=DeepSeek-14B2025.07 | 55.2 | |
| Prefix TuningPretrained Model=DeepSeek-32B, m=322025.07 | 53 | |
| EMPOBase Model=OLMO3-7B-Base2026.04 | 52.9 | |
| ICLPretrained Model=Mistral-12B2025.07 | 51.3 | |
| Prefix TuningPretrained Model=DeepSeek-14B, m=322025.07 | 49.8 | |
| Prefix TuningPretrained Model=Qwen3-14B, m=322025.07 | 49.3 | |
| Zero-RL (PPO)Base Model=OLMO3-7B-Base2026.04 | 46.8 | |
| TTRLBase Model=OLMO3-7B-Base2026.04 | 45.4 | |
| MobileMoE-LActive Parameters=922M, Total Parameters=5.3B, Few-shot count=3-shot2026.05 | 37.8 | |
| MobileMoE-MActive Parameters=528M, Total Parameters=2.8B, Few-shot count=3-shot2026.05 | 37.7 | |
| Prefix TuningPretrained Model=Mistral-12B, m=322025.07 | 37 | |
| MobileMoE-SActive Parameters=272M, Total Parameters=1.3B, Few-shot count=3-shot2026.05 | 31.8 |