Commonsense Inference on HellaSwag
88.97AccuracyN-3-Super 120B-A12B-Base
Evaluation Results
| Method | Links | |
|---|---|---|
| N-3-Super 120B-A12B-BaseShots=102026.04 | 88.97 | |
| GLM-4.5 Air-BaseShots=102026.04 | 87.7 | |
| Llama 3.1-70BPrecision=BF162026.02 | 85.75 | |
| Attn-QATModel=Llama 3.1-70B, Precision=Attn-QAT2026.02 | 85.57 | |
| AceGPT-v2-70B-ChatSize=70B, Few-shot settings=0-shot2026.03 | 85.53 | |
| Fanar-27BSize=27B, Few-shot settings=0-shot2026.03 | 85.32 | |
| Ling-flash base-2.0Shots=102026.04 | 84.69 | |
| Llama 3.1-70BPrecision=FP42026.02 | 84.63 | |
| Jais-2-70B-ChatSize=70B, Few-shot settings=0-shot2026.03 | 84.55 | |
| Llama-3.3-70b-InstructSize=70B, Few-shot settings=0-shot2026.03 | 84.38 | |
| Gemma-3-27B-itSize=27B, Few-shot settings=0-shot2026.03 | 84.15 | |
| AceGPT-v2-32B-ChatSize=32B, Few-shot settings=0-shot2026.03 | 83.32 | |
| Fanar-1-9b-instructSize=9B, Few-shot settings=0-shot2026.03 | 83.01 | |
| Qwen3-32BSize=32B, Few-shot settings=0-shot2026.03 | 82.7 | |
| Qwen3-14BPrecision=BF162026.02 | 81.4 | |
| Qwen3-14BPrecision=FP42026.02 | 80.5 | |
| Attn-QATModel=Qwen3-14B, Precision=Attn-QAT2026.02 | 80.34 | |
| RTNFormat=INT82025.09 | 80.2 | |
| FP16Format=FP162025.09 | 80 | |
| GPTQFormat=INT82025.09 | 80 | |
| GPTQFormat=FP82025.09 | 79.9 | |
| RTNFormat=FP82025.09 | 79.8 | |
| MR-GPTQFormat=NVINT42025.09 | 79.14 | |
| Allam-7B-Instruct-preview-v2Size=7B, Few-shot settings=0-shot2026.03 | 79.06 | |
| QAT+HadFormat=NVFP2025.09 | 79 | |
| QATFormat=NVFP2025.09 | 78.9 | |
| OursQBackbone=Qwen3-MoE-30B-A3B, Type=P25% Q4b, Storage=11.64GB, Evaluation Protocol=zero-shot2026.06 | 78.87 | |
| GPTQFormat=NVFP2025.09 | 78.7 | |
| RTNFormat=NVINT42025.09 | 78.64 | |
| BaselineBackbone=Qwen3-MoE-30B-A3B, Type=–, Storage=61.06GB, Evaluation Protocol=zero-shot2026.06 | 78.5 | |
| RTNFormat=NVFP2025.09 | 78.4 | |
| GPTQ+HadFormat=NVFP2025.09 | 78.4 | |
| RTN + HTFormat=NVINT42025.09 | 78.38 | |
| GPTQ+Had128Format=NVFP2025.09 | 78.3 | |
| QAT+HadFormat=MXFP2025.09 | 78.3 | |
| QATFormat=MXFP2025.09 | 77.6 | |
| GPTQFormat=NVINT42025.09 | 77.59 | |
| RTN+Had128Format=NVFP2025.09 | 77.5 | |
| RTN+HadFormat=NVFP2025.09 | 77.3 | |
| OursBackbone=Qwen3-MoE-30B-A3B, Type=P50%, Storage=32.07GB, Evaluation Protocol=zero-shot2026.06 | 77.19 | |
| BaselineBackbone=Qwen1.5-MoE-A2.7B, Type=–, Storage=28.67GB, Evaluation Protocol=zero-shot2026.06 | 77.17 | |
| GPTQ+HadFormat=MXFP2025.09 | 77 | |
| FPModel=LLaDA-1.5, Quantization=Full Precision2026.06 | 76.91 | |
| GPTQ+Had128Format=MXFP2025.09 | 76.9 | |
| FPModel=LLaDA-Instruct, Quantization=Full Precision2026.06 | 76.89 | |
| MR-GPTQFormat=MXINT42025.09 | 76.63 | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P25%, Storage=28.67GB, Evaluation Protocol=zero-shot2026.06 | 76.48 | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=7.10GB, Evaluation Protocol=zero-shot2026.06 | 76.48 | |
| RTN+Had128Format=MXFP2025.09 | 76 | |
| GPTQFormat=MXFP2025.09 | 75.52 | |
| RTN+HadFormat=MXFP2025.09 | 75.2 | |
| GPTQFormat=MXINT42025.09 | 75.04 | |
| OursQBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=zero-shot2026.06 | 75.01 | |
| EAC-MoEBackbone=Qwen1.5-MoE-A2.7B, Type=P38% Q3.03b, Storage=4.35GB, Evaluation Protocol=zero-shot2026.06 | 74.97 | |
| BF16Rate=16.000, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 74.9 | |
| KarnakSize=40B, Few-shot settings=0-shot2026.03 | 74.9 | |
| Huffman-GPTQRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 74.84 | |
| HRTNRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 74.72 | |
| WaterSICRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 74.6 | |
| FAIR-CalibModel=LLaDA-Instruct, Quantization=W4A42026.06 | 74.5 | |
| RTNRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 74.3 | |
| FAIR-CalibModel=LLaDA-1.5, Quantization=W4A42026.06 | 74.29 | |
| GPTQRate=4.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 74.22 | |
| FlatQuantModel=LLaDA-Instruct, Quantization=W4A42026.06 | 74.15 | |
| FlatQuantModel=LLaDA-1.5, Quantization=W4A42026.06 | 73.97 | |
| QuaRotModel=LLaDA-1.5, Quantization=W4A42026.06 | 73.96 | |
| RTN + HTFormat=MXINT42025.09 | 73.94 | |
| FPModel=Dream-Instruct, Quantization=Full Precision2026.06 | 73.89 | |
| RTNFormat=MXFP2025.09 | 73.8 | |
| FPModel=Dream-Base, Quantization=Full Precision2026.06 | 73.65 | |
| WaterSICRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 73.38 | |
| QuaRotModel=LLaDA-Instruct, Quantization=W4A42026.06 | 73.27 | |
| Huffman-GPTQRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 73.18 | |
| OursBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=16.17GB, Evaluation Protocol=zero-shot2026.06 | 73.11 | |
| FPModel=LLaDA-Base, Quantization=Full Precision2026.06 | 72.9 | |
| HRTNRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 72.36 | |
| RTNFormat=MXINT42025.09 | 72.06 | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=28.67GB, Evaluation Protocol=zero-shot2026.06 | 71.92 | |
| FAIR-CalibModel=LLaDA-Base, Quantization=W4A42026.06 | 71.27 | |
| He et al.Backbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=zero-shot2026.06 | 71.22 | |
| FlatQuantModel=LLaDA-Base, Quantization=W4A42026.06 | 71.15 | |
| FAIR-CalibModel=Dream-Instruct, Quantization=W4A42026.06 | 70.98 | |
| GPTQRate=3.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 70.8 | |
| MoEITSBackbone=DeepSeek-V2 Lite, Evaluation setting=15-shot, Average Accuracy (Av)=67.99, Parameter Reduction (P)=55.42%2026.04 | 70.8 | |
| FAIR-CalibModel=Dream-Base, Quantization=W4A42026.06 | 70.6 | |
| Gemma3-4BModel Scale=4B2026.04 | 70.26 | |
| QuaRotModel=LLaDA-Base, Quantization=W4A42026.06 | 69.96 | |
| FlatQuantModel=Dream-Instruct, Quantization=W4A42026.06 | 69.82 | |
| FlatQuantModel=Dream-Base, Quantization=W4A42026.06 | 69.54 | |
| Llama3.2-3BModel Scale=3B2026.04 | 69.32 | |
| Qwen3-4BModel Scale=4B2026.04 | 67.92 | |
| Qwen2.5-3BModel Scale=3B2026.04 | 67.71 | |
| QuaRotModel=Dream-Instruct, Quantization=W4A42026.06 | 67.28 | |
| QuaRotModel=Dream-Base, Quantization=W4A42026.06 | 66.92 | |
| RTNModel=LLaDA-1.5, Quantization=W4A42026.06 | 66.53 | |
| SpB2.0-5BModel Scale=5B, Additional Training Tokens=14B2026.04 | 66.41 | |
| RTNModel=LLaDA-Instruct, Quantization=W4A42026.06 | 65.93 | |
| WaterSICRate=2.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 64.52 | |
| Huffman-GPTQRate=2.125, Evaluation Protocol=Zero-shot, Backbone Model=Qwen3-8B2026.03 | 63.86 | |
| MoNEBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=zero-shot2026.06 | 63.06 |