List your company
ResearchFormerly WizWand

Commonsense Reasoning on ARC-C (Accuracy)

65.27AccuracyQwen3-235B-A22B
26.675636.695346.71556.7347Jun 10, 2026Jun 14, 2026Jun 19, 2026Jun 23, 2026Jun 28, 2026Jul 2, 2026Jul 7, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Qwen3-235B-A22B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0665.27
MoE with MPIModel Scale=11B, Setup=5-shot2026.0662.24
MoEModel Scale=11B, Setup=5-shot2026.0661.54
RF2-100B-A6.1B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0661.01
Qwen3-32B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0660.92
Ministral3-8BGen. Mode=AR, Avg TPF=1.002026.0760.75
Qwen3-14B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0660.58
Dream-7BGen. Mode=Diff., Avg TPF=1.002026.0759.13
MoE with MPIModel Scale=3B, Setup=5-shot2026.0658.96
Nemotron-Labs-Diffusion-8BGen. Mode=AR, Avg TPF=1.002026.0758.11
Nemotron-Labs-Diffusion-8BGen. Mode=Diff., Avg TPF=2.062026.0758.11
Nemotron-Labs-Diffusion-8BGen. Mode=Linear SS, Avg TPF=4.672026.0758.11
Nemotron-Labs-Diffusion-8BGen. Mode=Quad. SS, Avg TPF=7.042026.0758.11
Llama2-70B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0657.59
Qwen3-8B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0656.57
Qwen3-30B-A3B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0656.23
MoEModel Scale=3B, Setup=5-shot2026.0655.91
Qwen3-4B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0653.84
CAT-QBase Model=Qwen3-32B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0653.41
Llama-3.1-8BGen. Mode=AR, Avg TPF=1.002026.0753.41
CAT-QBase Model=Llama2-70B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0653.24
Qwen3-8BGen. Mode=AR, Avg TPF=1.002026.0753.16
LLaDA-8BGen. Mode=Diff., Avg TPF=1.002026.0749.15
CAT-QBase Model=Qwen3-235B-A22B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0648.21
Llama2-7B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0646.42
CAT-QBase Model=Qwen3-14B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0645.22
CAT-QBase Model=RF2-100B-A6.1B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0645.05
CAT-QBase Model=Qwen3-30B-A3B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.0643.17
Qwen3-1.7B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0642.66
CAT-QBase Model=Qwen3-8B, #Bits=W1.58A8, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0642.49
CAT-QBase Model=Qwen3-8B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0640.7
CAT-QBase Model=Qwen3-4B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0636.95
CAT-QBase Model=Qwen3-4B, #Bits=W1.58A8, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0635.69
CAT-QBase Model=Llama2-7B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0633.62
CAT-QBase Model=Qwen3-1.7B, #Bits=W1.58A8, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0628.67
CAT-QBase Model=Qwen3-1.7B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.0628.16