List your company
ResearchFormerly WizWand

Common-sense Reasoning on ARC easy

93.27ARC (easy) AccuracyHiRA
51.025261.992672.9683.9274Feb 19, 2024Jul 9, 2024Nov 28, 2024Apr 18, 2025Sep 7, 2025Jan 26, 2026Jun 17, 2026
Updated 3mo ago

Evaluation results

MethodLinks
HiRAModel=Llama-3-8B, Params (%)=0.70022026.0493.27——
BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.0491.2——
MoRAModel=Llama-3-8B, Params (%)=0.69972026.0491.16——
DoRAModel=Llama-3-8B, Params (%)=0.70022026.0490.5——
ChatGPTModel=ChatGPT2026.0489.8——
HiRAModel=Llama-2-7B, Params (%)=0.82562026.0486.74——
BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.0485.56——
MoRAModel=Llama-2-7B, Params (%)=0.82412026.0485.31——
LoRAModel=Llama-3-8B, Params (%)=0.70022026.0484.2——
VanillaBackbone=Mixtral-8x7B-v0.1, Sparsity=0%2025.1184.1——
DoRAModel=Llama-2-7B, Params (%)=0.82562026.0483.7——
PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1183.2——
PuzzleMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1182.4——
NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1182——
STUNBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1181.8——
D2Backbone=Mixtral-8x7B-v0.1, Sparsity=20%2025.1180——
Sub-MoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1180——
LoRAModel=Llama-2-7B, Params (%)=0.82562026.0479.8——
HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=25%2025.1179.3——
VanillaBackbone=Qwen3-MoE-30B-A3B, Sparsity=0%2025.1179.3——
PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1178.9——
WandaBackbone=Mixtral-8x7B-v0.1, Sparsity=2:42025.1178.8——
NAEEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1178.5——
PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1178.5——
D2Backbone=Mixtral-8x7B-v0.1, Sparsity=40%2025.1178——
WandaBackbone=Qwen3-MoE-30B-A3B, Sparsity=2:42025.1176.1——
VanillaBackbone=Deepseek-MoE-16b, Sparsity=0%2025.1175.9——
PuzzleMoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.1175.7——
PuzzleMoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.1175.2——
Sub-MoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1175——
D2Backbone=Deepseek-MoE-16b, Sparsity=20%2025.1174——
PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=50%2025.1173.5——
PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.1173.4——
VanillaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=0%2025.1173.2——
CleanTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0572.5——
CleanTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0572.5——
WandaBackbone=Qwen1.5-MoE-A2.7B, Sparsity=2:42025.1172.2——
HC-SMoEBackbone=Mixtral-8x7B-v0.1, Sparsity=50%2025.1172——
Sub-MoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.1172——
WandaBackbone=Deepseek-MoE-16b, Sparsity=2:42025.1171.2——
HC-SMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1170.8——
D2Backbone=Qwen3-MoE-30B-A3B, Sparsity=20%2025.1170.6——
Sub-MoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=25%2025.1170——
Mamba2Number of Parameters=1.3B, Zero-shot=true2026.0369.87——
ROMERTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0569.4——
D2Backbone=Deepseek-MoE-16b, Sparsity=40%2025.1169——
Sub-MoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1169——
HADESNumber of Parameters=1B, Zero-shot=true2026.0368.48——
D2Backbone=Qwen1.5-MoE-A2.7B, Sparsity=20%2025.1168.3——
Weight-remappingTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0568.2——
Palimpsa-M (Fine-tuned 2BT)Model Scale=760M2026.0267.97——
DeltaNetNumber of Parameters=1.3B, Zero-shot=true2026.0367.63——
HC-SMoEBackbone=Qwen1.5-MoE-A2.7B, Sparsity=25%2025.1167.5——
ROMERTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0566.9——
Bit-slicingTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0566.4——
AverageTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0566——
k-b calibrationTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0565.8——
D2Backbone=Qwen3-MoE-30B-A3B, Sparsity=40%2025.1165.2——
HC-SMoEBackbone=Deepseek-MoE-16b, Sparsity=25%2025.1165.1——
Weight-remappingTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0564.7——
VanillaArchitecture Category=State Space Model, Base Model=Mamba2, Evaluation Protocol=zero-shot2025.0964.3——
VanillaTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.0564.2——
StateXArchitecture Category=State Space Model, Base Model=Mamba2, Evaluation Protocol=zero-shot2025.0964——
RetNetNumber of Parameters=1.3B, Zero-shot=true2026.0363.68——
SwiAttnAttention Mechanism=Switch Attention2026.0363——
Sub-MoEBackbone=Deepseek-MoE-16b, Sparsity=50%2025.1163——
SWA-ZSAttention Mechanism=Sliding Window Attention, Training/Evaluation Protocol=Zero-shot2026.0362.9——
FullAttnAttention Mechanism=Full attention2026.0362.9——
SWA-CPTAttention Mechanism=Sliding Window Attention, Training/Evaluation Protocol=Continual pretraining2026.0362.7——
StaticHybridAttention Mechanism=Static hybrid of full attention and SWA, Ratio (Full:SWA)=1:32026.0362.3——
GSAState size=128 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.0961.9——
Bit-slicingTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0561——
HC-SMoEBackbone=Qwen3-MoE-30B-A3B, Sparsity=50%2025.1160.9——
HGRN2State size=128 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.0960.8——
MambaState size=64 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.0960.7——
AverageTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0560.6——
k-b calibrationTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0560.4——
FP-16Backbone=LLaMA-1-13B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.0259.85——
Xfmr++State size=N/A, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.0959.8——
RetNetState size=512 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.0959.6——
GLAState size=256 × Ld, Model size=2.7B, Training tokens=100B, Number of layers (L)=32, Model dimension (d)=2,560, Evaluation protocol=Zero-shot2024.0959.2——
FP-16Backbone=LLaMA-1-30B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.0258.92——
FP-16Backbone=LLaMA-1-65B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.0258.75——
HGRN2State size=128 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.0958.1——
GSAState size=128 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.0958.1——
Palimpsa-D (Fine-tuned 1BT)Model Scale=170M2026.0258.04——
VanillaTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.0557.5——
RetNetState size=512 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.0957.4——
MambaState size=64 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.0957.1——
GLAState size=256 × Ld, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.0955.4——
VanillaArchitecture Category=Linear Attention, Base Model=GLA, Evaluation Protocol=zero-shot2025.0954.5——
EFLAParameters=1.3B, Zero-shot=true2025.1254.4——
RegMix-DProxy models=512, Mode=Online2026.0654.27——
Xfmr++State size=N/A, Model size=1.3B, Training tokens=100B, Number of layers (L)=24, Model dimension (d)=2,048, Evaluation protocol=Zero-shot2024.0954.1——
RegMix-DProxy models=128, Mode=Online2026.0653.93——
StateXArchitecture Category=Linear Attention, Base Model=GLA, Evaluation Protocol=zero-shot2025.0953.9——
DB-LLMBackbone=LLaMA-1-65B, Bit-width=W2A16, Evaluation Protocol=zero-shot2024.0253.66——
RegMix-DProxy models=512, Mode=Offline2026.0653.49——
RegMix-DProxy models=128, Mode=Offline2026.0653.12——
OmniQuantBackbone=LLaMA-1-65B, Bit-width=W2A16, Evaluation Protocol=zero-shot2024.0252.65——