List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on ARC Challenge (Non-generative Acc)

30Non-generative AccuracyNoPE
-0.958727.0786415.11623.15336Nov 21, 2025Nov 23, 2025Nov 25, 2025Nov 28, 2025Nov 30, 2025Dec 2, 2025Dec 5, 2025
Updated 6mo ago

Evaluation results

MethodLinks
NoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1130
Selective RoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1128.8
RoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1128.5
Selective RoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1126.6
Selective RoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1125.7
Selective RoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B, Variant=stable_selective_rope2025.1125.5
NoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1125
NoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1124.9
RoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1124.6
RoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1124.5
NoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B2025.1124.2
RoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B2025.1124.1
BF16Model=Llama-3 70B, Setting=W16A162025.120.6451
SpQRModel=Llama-3 70B, Setting=W4A162025.120.6399
SQ-formatModel=Llama-3 70B, Setting=W4A(SQ6)2025.120.628
GPTQModel=Llama-3 70B, Setting=W4A82025.120.6254
SpinQuantModel=Llama-3 70B, Setting=W4A82025.120.6203
SQ-formatModel=Llama-3 70B, Setting=W4A(SQ5)2025.120.6195
SQ-formatModel=Llama-3 70B, Setting=W(SQ6)A42025.120.6101
SQ-formatModel=Llama-3 70B, Setting=W(SQ5)A42025.120.5998
GPTQModel=Llama-3 70B, Setting=W4A42025.120.5989
SQ-formatModel=Llama-3 70B, Setting=W(SQ4.5)A42025.120.5964
BF16Model=Qwen-3 30B-A3B, Setting=W16A162025.120.558
SQ-formatModel=Qwen-3 30B-A3B, Setting=W4A(SQ5)2025.120.5546
SQ-formatModel=Qwen-3 30B-A3B, Setting=W4A(SQ6)2025.120.5538
SQ-formatModel=Qwen-3 30B-A3B, Setting=W(SQ6)A42025.120.5375
BF16Model=Llama-3 8B, Setting=W16A162025.120.5341
GPTQModel=Llama-3 8B, Setting=W4A82025.120.5315
GPTQModel=Qwen-3 30B-A3B, Setting=W4A82025.120.5273
SpQRModel=Llama-3 8B, Setting=W4A162025.120.5213
SQ-formatModel=Llama-3 8B, Setting=W4A(SQ6)2025.120.5213
SQ-formatModel=Llama-3 8B, Setting=W4A(SQ5)2025.120.5196
SpinQuantModel=Llama-3 8B, Setting=W4A82025.120.5178
GPTQModel=Qwen-3 30B-A3B, Setting=W4A42025.120.4948
SQ-formatModel=Llama-3 8B, Setting=W(SQ6)A42025.120.494
SparseGPTModel=Llama-3 70B, Setting=W16A162025.120.4872
SpinQuantModel=Llama-3 8B, Setting=W4A42025.120.4795
SQ-formatModel=Llama-3 8B, Setting=W(SQ4.5)A42025.120.4761
SQ-formatModel=Llama-3 8B, Setting=W(SQ5)A42025.120.4744
GPTQModel=Llama-3 8B, Setting=W4A42025.120.4692
SmoothQuantModel=Qwen-3 30B-A3B, Setting=W4A82025.120.4035
SpinQuantModel=Llama-3 70B, Setting=W4A42025.120.3797
SmoothQuantModel=Llama-3 8B, Setting=W4A82025.120.372
SparseGPTModel=Llama-3 8B, Setting=W16A162025.120.3174
SmoothQuantModel=Llama-3 70B, Setting=W4A42025.120.2521
SmoothQuantModel=Llama-3 70B, Setting=W4A82025.120.244
SmoothQuantModel=Llama-3 8B, Setting=W4A42025.120.2423
SmoothQuantModel=Qwen-3 30B-A3B, Setting=W4A42025.120.232