List your company
ResearchFormerly WizWand

Commonsense Inference on HellaSwag

88.97AccuracyN-3-Super 120B-A12B-Base
62.023669.019376.01583.0107Sep 27, 2025Nov 9, 2025Dec 23, 2025Feb 5, 2026Mar 20, 2026May 3, 2026Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
N-3-Super 120B-A12B-BaseShots=102026.0488.97
GLM-4.5 Air-BaseShots=102026.0487.7
Llama 3.1-70BPrecision=BF162026.0285.75
Attn-QATModel=Llama 3.1-70B, Precision=Attn-QAT2026.0285.57
AceGPT-v2-70B-ChatSize=70B, Few-shot settings=0-shot2026.0385.53
Fanar-27BSize=27B, Few-shot settings=0-shot2026.0385.32
Ling-flash base-2.0Shots=102026.0484.69
Llama 3.1-70BPrecision=FP42026.0284.63
Jais-2-70B-ChatSize=70B, Few-shot settings=0-shot2026.0384.55
Llama-3.3-70b-InstructSize=70B, Few-shot settings=0-shot2026.0384.38
Gemma-3-27B-itSize=27B, Few-shot settings=0-shot2026.0384.15
AceGPT-v2-32B-ChatSize=32B, Few-shot settings=0-shot2026.0383.32
Fanar-1-9b-instructSize=9B, Few-shot settings=0-shot2026.0383.01
Qwen3-32BSize=32B, Few-shot settings=0-shot2026.0382.7
Qwen3-14BPrecision=BF162026.0281.4
Qwen3-14BPrecision=FP42026.0280.5
Attn-QATModel=Qwen3-14B, Precision=Attn-QAT2026.0280.34
RTNFormat=INT82025.0980.2
FP16Format=FP162025.0980
GPTQFormat=INT82025.0980
GPTQFormat=FP82025.0979.9
RTNFormat=FP82025.0979.8
MR-GPTQFormat=NVINT42025.0979.14
Allam-7B-Instruct-preview-v2Size=7B, Few-shot settings=0-shot2026.0379.06
QAT+HadFormat=NVFP2025.0979
QATFormat=NVFP2025.0978.9
OursQBackbone=Qwen3-MoE-30B-A3B, Type=P25% Q4b, Storage=11.64GB, Evaluation Protocol=zero-shot2026.0678.87
GPTQFormat=NVFP2025.0978.7
RTNFormat=NVINT42025.0978.64
BaselineBackbone=Qwen3-MoE-30B-A3B, Type=–, Storage=61.06GB, Evaluation Protocol=zero-shot2026.0678.5
RTNFormat=NVFP2025.0978.4
GPTQ+HadFormat=NVFP2025.0978.4
RTN + HTFormat=NVINT42025.0978.38
GPTQ+Had128Format=NVFP2025.0978.3
QAT+HadFormat=MXFP2025.0978.3
QATFormat=MXFP2025.0977.6
GPTQFormat=NVINT42025.0977.59
RTN+Had128Format=NVFP2025.0977.5
RTN+HadFormat=NVFP2025.0977.3
OursBackbone=Qwen3-MoE-30B-A3B, Type=P50%, Storage=32.07GB, Evaluation Protocol=zero-shot2026.0677.19
BaselineBackbone=Qwen1.5-MoE-A2.7B, Type=–, Storage=28.67GB, Evaluation Protocol=zero-shot2026.0677.17
GPTQ+HadFormat=MXFP2025.0977
FPModel=LLaDA-1.5, Quantization=Full Precision2026.0676.91
GPTQ+Had128Format=MXFP2025.0976.9
FPModel=LLaDA-Instruct, Quantization=Full Precision2026.0676.89
MR-GPTQFormat=MXINT42025.0976.63
WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P25%, Storage=28.67GB, Evaluation Protocol=zero-shot2026.0676.48
WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=7.10GB, Evaluation Protocol=zero-shot2026.0676.48
RTN+Had128Format=MXFP2025.0976
GPTQFormat=MXFP2025.0975.52
RTN+HadFormat=MXFP2025.0975.2
GPTQFormat=MXINT42025.0975.04
OursQBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=zero-shot2026.0675.01
EAC-MoEBackbone=Qwen1.5-MoE-A2.7B, Type=P38% Q3.03b, Storage=4.35GB, Evaluation Protocol=zero-shot2026.0674.97
BF16Rate=16.000, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0374.9
KarnakSize=40B, Few-shot settings=0-shot2026.0374.9
Huffman-GPTQRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0374.84
HRTNRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0374.72
WaterSICRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0374.6
FAIR-CalibModel=LLaDA-Instruct, Quantization=W4A42026.0674.5
RTNRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0374.3
FAIR-CalibModel=LLaDA-1.5, Quantization=W4A42026.0674.29
GPTQRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0374.22
FlatQuantModel=LLaDA-Instruct, Quantization=W4A42026.0674.15
FlatQuantModel=LLaDA-1.5, Quantization=W4A42026.0673.97
QuaRotModel=LLaDA-1.5, Quantization=W4A42026.0673.96
RTN + HTFormat=MXINT42025.0973.94
FPModel=Dream-Instruct, Quantization=Full Precision2026.0673.89
RTNFormat=MXFP2025.0973.8
FPModel=Dream-Base, Quantization=Full Precision2026.0673.65
WaterSICRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0373.38
QuaRotModel=LLaDA-Instruct, Quantization=W4A42026.0673.27
Huffman-GPTQRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0373.18
OursBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=16.17GB, Evaluation Protocol=zero-shot2026.0673.11
FPModel=LLaDA-Base, Quantization=Full Precision2026.0672.9
HRTNRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0372.36
RTNFormat=MXINT42025.0972.06
WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=28.67GB, Evaluation Protocol=zero-shot2026.0671.92
FAIR-CalibModel=LLaDA-Base, Quantization=W4A42026.0671.27
He et al.Backbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=zero-shot2026.0671.22
FlatQuantModel=LLaDA-Base, Quantization=W4A42026.0671.15
FAIR-CalibModel=Dream-Instruct, Quantization=W4A42026.0670.98
GPTQRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0370.8
MoEITSBackbone=DeepSeek-V2 Lite, Evaluation setting=15-shot, Average Accuracy (Av)=67.99, Parameter Reduction (P)=55.42%2026.0470.8
FAIR-CalibModel=Dream-Base, Quantization=W4A42026.0670.6
Gemma3-4BModel Scale=4B2026.0470.26
QuaRotModel=LLaDA-Base, Quantization=W4A42026.0669.96
FlatQuantModel=Dream-Instruct, Quantization=W4A42026.0669.82
FlatQuantModel=Dream-Base, Quantization=W4A42026.0669.54
Llama3.2-3BModel Scale=3B2026.0469.32
Qwen3-4BModel Scale=4B2026.0467.92
Qwen2.5-3BModel Scale=3B2026.0467.71
QuaRotModel=Dream-Instruct, Quantization=W4A42026.0667.28
QuaRotModel=Dream-Base, Quantization=W4A42026.0666.92
RTNModel=LLaDA-1.5, Quantization=W4A42026.0666.53
SpB2.0-5BModel Scale=5B, Additional Training Tokens=14B2026.0466.41
RTNModel=LLaDA-Instruct, Quantization=W4A42026.0665.93
WaterSICRate=2.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0364.52
Huffman-GPTQRate=2.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.0363.86
MoNEBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=zero-shot2026.0663.06