List your company
ResearchFormerly WizWand

Common Sense Reasoning on ARC-Challenge (0-shot)

57.2Accuracyfp16
17.26427.6323848.368Oct 30, 2025Nov 17, 2025Dec 5, 2025Dec 23, 2025Jan 10, 2026Jan 28, 2026Feb 16, 2026
Updated 6mo ago

Evaluation results

MethodLinks
fp16Size=70B2026.0257.2
MagRSize=70B2026.0256.2
AstroSize=70B2026.0254.8
OmniQuantSize=70B2026.0254.3
fp16Size=13B2026.0248.8
MagRSize=13B2026.0247.4
OmniQuantSize=13B2026.0246.5
AstroSize=13B2026.0246.2
Llama2-7BBackbone=Llama2-7B2026.0245.5
fp16Size=7B2026.0245.1
AstroSize=7B2026.0243.3
OmniQuantSize=7B2026.0242.2
MagRSize=7B2026.0240.5
Gated DeltaNet-H1Setting=Zero-shot, Model Size=1.3B parameters2025.1040.1
COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.22026.0239.6
Nirvana (Ours)Setting=Zero-shot, Model Size=1.3B parameters2025.1039.51
Gated DeltaNet-H2Setting=Zero-shot, Model Size=1.3B parameters2025.1039.07
Gated DeltaNetSetting=Zero-shot, Model Size=1.3B parameters2025.1038.39
Mamba2Setting=Zero-shot, Model Size=1.3B parameters2025.1037.88
Nirvana-noTriggerSetting=Zero-shot, Model Size=1.3B parameters2025.1037.84
SambaSetting=Zero-shot, Model Size=1.3B parameters2025.1036.17
DeltaNetSetting=Zero-shot, Model Size=1.3B parameters2025.1035.66
MambaSetting=Zero-shot, Model Size=1.3B parameters2025.1035.4
HGRN2Setting=Zero-shot, Model Size=1.3B parameters2025.1035.32
Transformer++Setting=Zero-shot, Model Size=1.3B parameters2025.1035.1
RetNetSetting=Zero-shot, Model Size=1.3B parameters2025.1033.78
Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.22026.0229.3
COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.42026.0228.5
Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.62026.0224.7
Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.42026.0224.5
COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.62026.0218.8