Zero-shot Evaluation on ARC, HellaSwag, MMLU, PIQA, and WinoGrande
47.7ARC Challenge AccuracyCodeQuanthad
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CodeQuanthadModels=Qwen3-30B-A3B, Quantization Bit-width=A4W4, Quantization Granularity=Embedding-wise, Online Hadamard transform=true2026.04 | 47.7 | 69.7 | 69.1 | 67.9 | 73.9 | 63.5 | 65.3 | |
| DuQuantModels=Qwen3-30B-A3B, Quantization Bit-width=A4W4, Quantization Granularity=Embedding-wise, Online Hadamard transform=false2026.04 | 47.2 | 66.2 | 68.7 | 65.4 | 73.9 | 60.6 | 63.7 | |
| DuQuantModels=DeepSeek-V2-Lite, Quantization Bit-width=A4W4, Quantization Granularity=Embedding-wise, Online Hadamard transform=false2026.04 | 45.5 | 70.8 | 62.3 | 40 | 77.5 | 69.3 | 62.4 | |
| CodeQuanthadModels=DeepSeek-V2-Lite, Quantization Bit-width=A4W4, Quantization Granularity=Embedding-wise, Online Hadamard transform=true2026.04 | 44.5 | 72.3 | 72.7 | 45.4 | 78.2 | 64.4 | 62.9 | |
| SpinQuanthadModels=DeepSeek-V2-Lite, Quantization Bit-width=A4W4, Quantization Granularity=Embedding-wise, Online Hadamard transform=true2026.04 | 42.7 | 69.2 | 70.6 | 42.5 | 77.4 | 63.8 | 61 | |
| SpinQuanthadModels=Qwen3-30B-A3B, Quantization Bit-width=A4W4, Quantization Granularity=Embedding-wise, Online Hadamard transform=true2026.04 | 41.5 | 60 | 62.8 | 58.4 | 69.2 | 62.2 | 59 |