List your company
ResearchFormerly WizWand

Common Sense Reasoning on ARC-Easy (0-shot)

79.5Accuracyfp16
27.063240.676654.2967.9034Oct 30, 2025Dec 8, 2025Jan 17, 2026Feb 26, 2026Apr 6, 2026May 16, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
fp16Size=70B2026.0279.5
MagRSize=70B2026.0279.5
AstroSize=70B2026.0279.3
OmniQuantSize=70B2026.0278.5
fp16Size=13B2026.0276.6
MagRSize=13B2026.0275.6
AstroSize=13B2026.0274.7
fp16Size=7B2026.0273.8
OmniQuantSize=13B2026.0273.4
Mamba2Setting=Zero-shot, Model Size=1.3B parameters2025.1072.47
Gated DeltaNet-H1Setting=Zero-shot, Model Size=1.3B parameters2025.1071.75
Gated DeltaNet-H2Setting=Zero-shot, Model Size=1.3B parameters2025.1071.33
Gated DeltaNetSetting=Zero-shot, Model Size=1.3B parameters2025.1071.21
AstroSize=7B2026.0271.2
OmniQuantSize=7B2026.0271.1
TequilaLLM-3B#Tokens=10B, Quantization=W1.58A162026.0670.2
Nirvana (Ours)Setting=Zero-shot, Model Size=1.3B parameters2025.1069.92
MagRSize=7B2026.0269.7
Qwen3-8B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.0669.65
MambaSetting=Zero-shot, Model Size=1.3B parameters2025.1069.52
HGRN2Setting=Zero-shot, Model Size=1.3B parameters2025.1069.4
Transformer++Setting=Zero-shot, Model Size=1.3B parameters2025.1068.83
SambaSetting=Zero-shot, Model Size=1.3B parameters2025.1068.81
Nirvana-noTriggerSetting=Zero-shot, Model Size=1.3B parameters2025.1068.8
DeltaNetSetting=Zero-shot, Model Size=1.3B parameters2025.1068.47
RetNetSetting=Zero-shot, Model Size=1.3B parameters2025.1067.34
Qwen3-4B + CAT-Q#Tokens=2M, Quantization=W1.58A162026.0666.41
TriLM-3.9B#Tokens=300B, Quantization=W1.58A162026.0666
BitNetV1-3.9B#Tokens=100B, Quantization=W1.58A162026.0664.2
Qwen3-4B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.0662.29
TernaryLLM-8B+KD#Tokens=1T, Quantization=W1.58A162026.0661.2
BitNetV1-7B#Tokens=100B, Quantization=W1.58A162026.0659.51
TriLM-1.5B#Tokens=300B, Quantization=W1.58A162026.0659
Original EndpointModel=1B, Zero-shot evaluation=true2026.0656.73
Qwen3-1.7B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.0655.88
BitNetV1-1.3B#Tokens=100B, Quantization=W1.58A162026.0654.9
Original PythiaModel=410M, Zero-shot evaluation=true2026.0651.89
Dual Learned MatchingModel=1B, Zero-shot evaluation=true2026.0637.16
Learned MatchingModel=1B, Zero-shot evaluation=true2026.0636.57
Dual Learned MatchingModel=410M, Zero-shot evaluation=true2026.0635.69
Learned MatchingModel=410M, Zero-shot evaluation=true2026.0633.25
Raw InterpolationModel=410M, Zero-shot evaluation=true2026.0629.71
Raw InterpolationModel=1B, Zero-shot evaluation=true2026.0629.08