Common-sense Reasoning on ARC easy
93.27ARC (easy) AccuracyHiRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HiRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 93.27 | — | — | |
| BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.04 | 91.2 | — | — | |
| MoRAModel=Llama-3-8B, Params (%)=0.69972026.04 | 91.16 | — | — | |
| DoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 90.5 | — | — | |
| ChatGPTModel=ChatGPT2026.04 | 89.8 | — | — | |
| HiRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 86.74 | — | — | |
| BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.04 | 85.56 | — | — | |
| MoRAModel=Llama-2-7B, Params (%)=0.82412026.04 | 85.31 | — | — | |
| LoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 84.2 | — | — | |
| VanillaBackbone=Mixtral-8x7B-v0.1, Sparsity=0%2025.11 | 84.1 | — | — | |
| DoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 83.7 | — | — | |
| PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 83.2 | — | — | |
| PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.11 | 82.4 | — | — | |
| NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 82 | — | — | |
| STUNBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 81.8 | — | — | |
| D2Backbone=Mixtral-8x7B-v0.1, Sparsity=20%2025.11 | 80 | — | — | |
| Sub-MoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 80 | — | — | |
| LoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 79.8 | — | — | |
| HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.11 | 79.3 | — | — | |
| VanillaBackbone=Qwen3-MoE-30B-A3B, Sparsity=0%2025.11 | 79.3 | — | — | |
| PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.11 | 78.9 | — | — | |
| WandaBackbone=Mixtral-8x7B-v0.1, Sparsity=2:42025.11 | 78.8 | — | — | |
| NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.11 | 78.5 | — | — | |
| PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.11 | 78.5 | — | — | |
| D2Backbone=Mixtral-8x7B-v0.1, Sparsity=40%2025.11 | 78 | — | — | |
| WandaBackbone=Qwen3-MoE-30B-A3B, Sparsity=2:42025.11 | 76.1 | — | — | |
| VanillaBackbone=Deepseek-MoE-16b, Sparsity=0%2025.11 | 75.9 | — | — | |
| PuzzleMoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.11 | 75.7 | — | — | |
| PuzzleMoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.11 | 75.2 | — | — | |
| Sub-MoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.11 | 75 | — | — | |
| D2Backbone=Deepseek-MoE-16b, Sparsity=20%2025.11 | 74 | — | — | |
| PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2025.11 | 73.5 | — | — | |
| PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.11 | 73.4 | — | — | |
| VanillaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=0%2025.11 | 73.2 | — | — | |
| CleanTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 72.5 | — | — | |
| CleanTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 72.5 | — | — | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=2:42025.11 | 72.2 | — | — | |
| HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.11 | 72 | — | — | |
| Sub-MoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.11 | 72 | — | — | |
| WandaBackbone=Deepseek-MoE-16b, Sparsity=2:42025.11 | 71.2 | — | — | |
| HC-SMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.11 | 70.8 | — | — | |
| D2Backbone=Qwen3-MoE-30B-A3B, Sparsity=20%2025.11 | 70.6 | — | — | |
| Sub-MoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.11 | 70 | — | — | |
| Mamba2Number of Parameters=1.3B, Zero-shot=true2026.03 | 69.87 | — | — | |
| ROMERTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 69.4 | — | — | |
| D2Backbone=Deepseek-MoE-16b, Sparsity=40%2025.11 | 69 | — | — | |
| Sub-MoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.11 | 69 | — | — | |
| HADESNumber of Parameters=1B, Zero-shot=true2026.03 | 68.48 | — | — | |
| D2Backbone=Qwen1.5-MoE-A2.7B, Sparsity=20%2025.11 | 68.3 | — | — | |
| Weight-remappingTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 68.2 | — | — | |
| Palimpsa-M (Fine-tuned 2BT)Model Scale=760M2026.02 | 67.97 | — | — | |
| DeltaNetNumber of Parameters=1.3B, Zero-shot=true2026.03 | 67.63 | — | — | |
| HC-SMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.11 | 67.5 | — | — | |
| ROMERTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 66.9 | — | — | |
| Bit-slicingTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 66.4 | — | — | |
| AverageTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 66 | — | — | |
| k-b calibrationTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 65.8 | — | — | |
| D2Backbone=Qwen3-MoE-30B-A3B, Sparsity=40%2025.11 | 65.2 | — | — | |
| HC-SMoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.11 | 65.1 | — | — | |
| Weight-remappingTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 64.7 | — | — | |
| VanillaArchitecture Category=State Space Model, Base Model=Mamba2, Evaluation Protocol=zero-shot2025.09 | 64.3 | — | — | |
| VanillaTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 64.2 | — | — | |
| StateXArchitecture Category=State Space Model, Base Model=Mamba2, Evaluation Protocol=zero-shot2025.09 | 64 | — | — | |
| RetNetNumber of Parameters=1.3B, Zero-shot=true2026.03 | 63.68 | — | — | |
| SwiAttnAttention Mechanism=Switch Attention2026.03 | 63 | — | — | |
| Sub-MoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.11 | 63 | — | — | |
| SWA-ZSAttention Mechanism=Sliding Window Attention, Training/Evaluation Protocol=Zero-shot2026.03 | 62.9 | — | — | |
| FullAttnAttention Mechanism=Full attention2026.03 | 62.9 | — | — | |
| SWA-CPTAttention Mechanism=Sliding Window Attention, Training/Evaluation Protocol=Continual pretraining2026.03 | 62.7 | — | — | |
| StaticHybridAttention Mechanism=Static hybrid of full attention and SWA, Ratio (Full:SWA)=1:32026.03 | 62.3 | — | — | |
| GSAState size=128 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.09 | 61.9 | — | — | |
| Bit-slicingTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 61 | — | — | |
| HC-SMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.11 | 60.9 | — | — | |
| HGRN2State size=128 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.09 | 60.8 | — | — | |
| MambaState size=64 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.09 | 60.7 | — | — | |
| AverageTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 60.6 | — | — | |
| k-b calibrationTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 60.4 | — | — | |
| FP-16Backbone=LLaMA-1-13B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.02 | 59.85 | — | — | |
| Xfmr++State size=N/A, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.09 | 59.8 | — | — | |
| RetNetState size=512 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.09 | 59.6 | — | — | |
| GLAState size=256 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.09 | 59.2 | — | — | |
| FP-16Backbone=LLaMA-1-30B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.02 | 58.92 | — | — | |
| FP-16Backbone=LLaMA-1-65B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.02 | 58.75 | — | — | |
| HGRN2State size=128 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.09 | 58.1 | — | — | |
| GSAState size=128 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.09 | 58.1 | — | — | |
| Palimpsa-D (Fine-tuned 1BT)Model Scale=170M2026.02 | 58.04 | — | — | |
| VanillaTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 57.5 | — | — | |
| RetNetState size=512 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.09 | 57.4 | — | — | |
| MambaState size=64 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.09 | 57.1 | — | — | |
| GLAState size=256 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.09 | 55.4 | — | — | |
| VanillaArchitecture Category=Linear Attention, Base Model=GLA, Evaluation Protocol=zero-shot2025.09 | 54.5 | — | — | |
| EFLAParameters=1.3B, Zero-shot=true2025.12 | 54.4 | — | — | |
| RegMix-DProxy models=512, Mode=Online2026.06 | 54.27 | — | — | |
| Xfmr++State size=N/A, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.09 | 54.1 | — | — | |
| RegMix-DProxy models=128, Mode=Online2026.06 | 53.93 | — | — | |
| StateXArchitecture Category=Linear Attention, Base Model=GLA, Evaluation Protocol=zero-shot2025.09 | 53.9 | — | — | |
| DB-LLMBackbone=LLaMA-1-65B, Bit-width=W2A16, Evaluation Protocol=zero-shot2024.02 | 53.66 | — | — | |
| RegMix-DProxy models=512, Mode=Offline2026.06 | 53.49 | — | — | |
| RegMix-DProxy models=128, Mode=Offline2026.06 | 53.12 | — | — | |
| OmniQuantBackbone=LLaMA-1-65B, Bit-width=W2A16, Evaluation Protocol=zero-shot2024.02 | 52.65 | — | — |