Mathematical Reasoning on GSM8K (200 samples)
66.5Exact Match ScoreGemma 3 12B IT (Base)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma 3 12B IT (Base)Mode=Base, Quantization=4-bit NF4, Max sequence length=512, Zero-shot=true, Instruction Prompt=Alpaca-style2026.04 | 66.5 | — | |
| Brainstacks (Routed)Mode=Routed, Quantization=4-bit NF4, Max sequence length=512, Zero-shot=true, Routing mechanism=meta-router selective gating, Chat floor=0.20, Gating=sigmoid, Instruction Prompt=Alpaca-style2026.04 | 66.5 | 0 | |
| LLaDA llada.cppSetting=llada.cpp2026.06 | 43.5 | — | |
| LLaDA (Ablation)Setting=NPU + MBSD + DPPR2026.06 | 43.5 | — | |
| LLaDA (Ablation)Setting=NPU2026.06 | 41.5 | — | |
| LLaDASetting=CPU2026.06 | 39 | — | |
| LLaDA (Ablation)Setting=CPU2026.06 | 39 | — | |
| LLaDA (Ablation)Setting=NPU + MBSD2026.06 | 37.5 | — | |
| LlamaSetting=NPU2026.06 | 31 | — | |
| LlamaSetting=CPU2026.06 | 30 | — |