Commonsense Reasoning on HellaSwag (Accuracy, Average Accuracy)
86.63AccuracySpikingBrain-76B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpikingBrain-76BParameters=12B/76B, Complexity Type=Hybrid, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 86.63 | — | |
| Qwen2.5Parameters=7B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 85.39 | — | |
| BF16Backbone=Llama-2-13B2026.05 | 79.38 | — | |
| Ministral3-8BGen. Mode=AR, Avg TPF=1.002026.07 | 79.01 | — | |
| Llama-3.1-8BGen. Mode=AR, Avg TPF=1.002026.07 | 78.93 | — | |
| Qwen3-8BGen. Mode=AR, Avg TPF=1.002026.07 | 78.59 | — | |
| TeacherNumber of layers=362026.05 | 76.95 | — | |
| Llama3Parameters=8B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 76.8 | — | |
| Nemotron-Labs-Diffusion-8BGen. Mode=AR, Avg TPF=1.002026.07 | 76.08 | — | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Diff., Avg TPF=2.062026.07 | 76.08 | — | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Linear SS, Avg TPF=4.672026.07 | 76.08 | — | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Quad. SS, Avg TPF=7.042026.07 | 76.08 | — | |
| BF16Backbone=LLaMA-2-7B2026.05 | 76 | — | |
| MixtralParameters=13B/47B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 75.63 | — | |
| Dream-7BGen. Mode=Diff., Avg TPF=1.002026.07 | 73.73 | — | |
| LLaDA-8BGen. Mode=Diff., Avg TPF=1.002026.07 | 71.05 | — | |
| SpikingBrain-7BParameters=7B, Complexity Type=Linear, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 68.95 | — | |
| Forward Matching + KLNumber of layers=242026.05 | 64.75 | — | |
| GNMR+Δ INT8/BF16Precision=Dynamic INT8/BF16, Model=LLaMA-2 13B2026.05 | 64.67 | — | |
| BF16Precision=BF16, Model=LLaMA-2 13B2026.05 | 64.62 | — | |
| Fixed INT8Precision=Fixed INT8, Model=LLaMA-2 13B2026.05 | 64.55 | — | |
| KL OnlyNumber of layers=242026.05 | 62.79 | — | |
| Oryx-TM (Mamba-2)Parameter Scale=1.4B, Family=Oryx-TM2026.05 | 62.1 | — | |
| Oryx-TG (Gated DeltaNet)Parameter Scale=1.4B, Family=Oryx-TG2026.05 | 62.1 | — | |
| Gated DeltaNetParameter Scale=1.4B, Family=Baseline2026.05 | 61.9 | — | |
| Oryx-TG (Transformer)Parameter Scale=1.4B, Family=Oryx-TG2026.05 | 61.8 | — | |
| Oryx-TM (Transformer)Parameter Scale=1.4B, Family=Oryx-TM2026.05 | 61.3 | — | |
| Mamba-2Parameter Scale=1.4B, Family=Baseline2026.05 | 60.9 | — | |
| TransformerParameter Scale=1.4B, Family=Baseline2026.05 | 60.6 | — | |
| OmniQuantBackbone=Llama-2-13B2026.05 | 60.38 | — | |
| LGPBackbone=Llama-2-13B2026.05 | 60 | — | |
| Oryx-TG (Gated DeltaNet)Parameter Scale=810M, Family=Oryx-TG2026.05 | 58.4 | — | |
| Gated DeltaNetParameter Scale=810M, Family=Baseline2026.05 | 58.3 | — | |
| Mamba-2Parameter Scale=810M, Family=Baseline2026.05 | 58.2 | — | |
| Oryx-TG (Transformer)Parameter Scale=810M, Family=Oryx-TG2026.05 | 58.1 | — | |
| Oryx-TM (Transformer)Parameter Scale=810M, Family=Oryx-TM2026.05 | 58 | — | |
| Oryx-TM (Mamba-2)Parameter Scale=810M, Family=Oryx-TM2026.05 | 57.9 | — | |
| SFT (no teacher)Number of layers=242026.05 | 57.2 | — | |
| TransformerParameter Scale=810M, Family=Baseline2026.05 | 57 | — | |
| Forward Matching + KLNumber of layers=182026.05 | 55.14 | — | |
| KL OnlyNumber of layers=182026.05 | 52.15 | — | |
| Mamba-2Parameter Scale=380M, Family=Baseline2026.05 | 51.8 | — | |
| Oryx-TG (Gated DeltaNet)Parameter Scale=380M, Family=Oryx-TG2026.05 | 51.5 | — | |
| Gated DeltaNetParameter Scale=380M, Family=Baseline2026.05 | 51.4 | — | |
| Oryx-TG (Transformer)Parameter Scale=380M, Family=Oryx-TG2026.05 | 51.3 | — | |
| Oryx-TM (Transformer)Parameter Scale=380M, Family=Oryx-TM2026.05 | 51.2 | — | |
| TransformerParameter Scale=380M, Family=Baseline2026.05 | 51 | — | |
| Oryx-TM (Mamba-2)Parameter Scale=380M, Family=Oryx-TM2026.05 | 51 | — | |
| LGPBackbone=LLaMA-2-7B2026.05 | 50.81 | — | |
| OmniQuantBackbone=LLaMA-2-7B2026.05 | 50.7 | — | |
| BF16Backbone=Qwen-3-0.6B2026.05 | 47.3 | — | |
| Muon32Model=LLaMA 1.1B, Evaluation Protocol=Zero-shot2026.05 | 45.9 | — | |
| SFT (no teacher)Number of layers=182026.05 | 42.99 | — | |
| Pruned (no training)Number of layers=242026.05 | 41.8 | — | |
| Muon8Model=LLaMA 1.1B, Evaluation Protocol=Zero-shot2026.05 | 41.2 | — | |
| GPTQBackbone=Llama-2-13B2026.05 | 41.08 | — | |
| Gated DeltaNetParameter Scale=130M, Family=Baseline2026.05 | 40.5 | — | |
| Oryx-TG (Gated DeltaNet)Parameter Scale=130M, Family=Oryx-TG2026.05 | 40.4 | — | |
| GRASPruneModel=LLaMA-2-13B, Parameter Retention=40%2026.04 | 40.32 | 44.12 | |
| MuonQ4Model=LLaMA 1.1B, Evaluation Protocol=Zero-shot2026.05 | 40.3 | — | |
| Mamba-2Parameter Scale=130M, Family=Baseline2026.05 | 40 | — | |
| Oryx-TG (Transformer)Parameter Scale=130M, Family=Oryx-TG2026.05 | 39.9 | — | |
| Oryx-TM (Mamba-2)Parameter Scale=130M, Family=Oryx-TM2026.05 | 39.8 | — | |
| Oryx-TM (Transformer)Parameter Scale=130M, Family=Oryx-TM2026.05 | 39.3 | — | |
| TransformerParameter Scale=130M, Family=Baseline2026.05 | 39.2 | — | |
| FLAPModel=LLaMA-2-13B, Parameter Retention=40%2026.04 | 38.18 | 43.44 | |
| Muon32Model=GPT-2 Large, Evaluation Protocol=Zero-shot2026.05 | 37.9 | — | |
| GPTQBackbone=LLaMA-2-7B2026.05 | 37.13 | — | |
| Muon8Model=GPT-2 Large, Evaluation Protocol=Zero-shot2026.05 | 36.2 | — | |
| Forward Matching + KLNumber of layers=92026.05 | 36.18 | — | |
| GRASPruneModel=Vicuna-7B, Parameter Retention=40%2026.04 | 35.14 | 41.61 | |
| GRASPruneModel=LLaMA-7B, Parameter Retention=40%2026.04 | 34.42 | 40.74 | |
| Muon32Model=LLaMA 350M, Evaluation Protocol=Zero-shot2026.05 | 34.3 | — | |
| MuonQ4Model=GPT-2 Large, Evaluation Protocol=Zero-shot2026.05 | 33.8 | — | |
| GRASPruneModel=LLaMA-2-7B, Parameter Retention=40%2026.04 | 33.75 | 40.6 | |
| KL OnlyNumber of layers=92026.05 | 33.75 | — | |
| Muon32Model=GPT-2 Medium, Evaluation Protocol=Zero-shot2026.05 | 33.1 | — | |
| LoopMDMS=12, Zero-shot=true2026.05 | 32.7 | — | |
| LoopMDMS=6, Zero-shot=true2026.05 | 32.5 | — | |
| Muon8Model=LLaMA 350M, Evaluation Protocol=Zero-shot2026.05 | 32.3 | — | |
| Muon8Model=GPT-2 Medium, Evaluation Protocol=Zero-shot2026.05 | 32.1 | — | |
| MDMZero-shot=true2026.05 | 31.9 | — | |
| LoopMDMS=1, Zero-shot=true2026.05 | 31.1 | — | |
| MuonQ4Model=LLaMA 350M, Evaluation Protocol=Zero-shot2026.05 | 31 | — | |
| FLAPModel=LLaMA-2-7B, Parameter Retention=40%2026.04 | 30.9 | 39.64 | |
| LLM-PrunerModel=LLaMA-2-13B, Parameter Retention=40%2026.04 | 30.26 | 38.81 | |
| MuonQ4Model=GPT-2 Medium, Evaluation Protocol=Zero-shot2026.05 | 30.2 | — | |
| FLAPModel=LLaMA-7B, Parameter Retention=40%2026.04 | 30.19 | 38.25 | |
| SliceGPTModel=LLaMA-2-13B, Parameter Retention=40%2026.04 | 29.59 | 37.25 | |
| SFT (no teacher)Number of layers=92026.05 | 29.53 | — | |
| LLM-PrunerModel=Vicuna-7B, Parameter Retention=40%2026.04 | 29.04 | 38.29 | |
| SliceGPTModel=Vicuna-7B, Parameter Retention=40%2026.04 | 28.73 | 36.88 | |
| SliceGPTModel=LLaMA-2-7B, Parameter Retention=40%2026.04 | 28.69 | 36.52 | |
| Muon4Model=LLaMA 1.1B, Evaluation Protocol=Zero-shot2026.05 | 28.5 | — | |
| LGPBackbone=Qwen-3-0.6B2026.05 | 28.44 | — | |
| OmniQuantBackbone=Qwen-3-0.6B2026.05 | 28.28 | — | |
| LLM-PrunerModel=LLaMA-7B, Parameter Retention=40%2026.04 | 27.66 | 37.58 | |
| Muon4Model=GPT-2 Large, Evaluation Protocol=Zero-shot2026.05 | 27.5 | — | |
| FLAPModel=Vicuna-7B, Parameter Retention=40%2026.04 | 27.4 | 38.99 | |
| Muon4Model=LLaMA 350M, Evaluation Protocol=Zero-shot2026.05 | 27 | — |