Commonsense Reasoning on ARC-C (Accuracy)
65.27AccuracyQwen3-235B-A22B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-235B-A22B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 65.27 | |
| MoE with MPIModel Scale=11B, Setup=5-shot2026.06 | 62.24 | |
| MoEModel Scale=11B, Setup=5-shot2026.06 | 61.54 | |
| RF2-100B-A6.1B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 61.01 | |
| Qwen3-32B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 60.92 | |
| Ministral3-8BGen. Mode=AR, Avg TPF=1.002026.07 | 60.75 | |
| Qwen3-14B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 60.58 | |
| Dream-7BGen. Mode=Diff., Avg TPF=1.002026.07 | 59.13 | |
| MoE with MPIModel Scale=3B, Setup=5-shot2026.06 | 58.96 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=AR, Avg TPF=1.002026.07 | 58.11 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Diff., Avg TPF=2.062026.07 | 58.11 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Linear SS, Avg TPF=4.672026.07 | 58.11 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Quad. SS, Avg TPF=7.042026.07 | 58.11 | |
| Llama2-70B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 57.59 | |
| Qwen3-8B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 56.57 | |
| Qwen3-30B-A3B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 56.23 | |
| MoEModel Scale=3B, Setup=5-shot2026.06 | 55.91 | |
| Qwen3-4B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 53.84 | |
| CAT-QBase Model=Qwen3-32B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 53.41 | |
| Llama-3.1-8BGen. Mode=AR, Avg TPF=1.002026.07 | 53.41 | |
| CAT-QBase Model=Llama2-70B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 53.24 | |
| Qwen3-8BGen. Mode=AR, Avg TPF=1.002026.07 | 53.16 | |
| LLaDA-8BGen. Mode=Diff., Avg TPF=1.002026.07 | 49.15 | |
| CAT-QBase Model=Qwen3-235B-A22B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 48.21 | |
| Llama2-7B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 46.42 | |
| CAT-QBase Model=Qwen3-14B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 45.22 | |
| CAT-QBase Model=RF2-100B-A6.1B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 45.05 | |
| CAT-QBase Model=Qwen3-30B-A3B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=MoE2026.06 | 43.17 | |
| Qwen3-1.7B#Bits=W16A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 42.66 | |
| CAT-QBase Model=Qwen3-8B, #Bits=W1.58A8, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 42.49 | |
| CAT-QBase Model=Qwen3-8B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 40.7 | |
| CAT-QBase Model=Qwen3-4B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 36.95 | |
| CAT-QBase Model=Qwen3-4B, #Bits=W1.58A8, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 35.69 | |
| CAT-QBase Model=Llama2-7B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 33.62 | |
| CAT-QBase Model=Qwen3-1.7B, #Bits=W1.58A8, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 28.67 | |
| CAT-QBase Model=Qwen3-1.7B, #Bits=W1.58A16, Regime=PTQ, Zero-shot=true, Architecture=Dense2026.06 | 28.16 |