Reasoning on Reasoning Benchmarks (Zero-shot)
69.99Overall Zero-Shot AccuracyMeta-Llama-3-8B Dense
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Meta-Llama-3-8B DenseZero-shot=true2026.03 | 69.99 | 80.79 | 79.17 | 77.69 | 53.33 | 72.85 | 69.68 | 45 | 81.44 | |
| 3BASiL-TMConfig=4:8+64LR, One-shot=true, Zero-shot=true2026.03 | 66.17 | 78.24 | 72.59 | 73.11 | 47.35 | 71.98 | 63.18 | 42.4 | 80.49 | |
| FP16 BaseBackbone=LLaMA3-3B, Weight bit-width (W)=16, Activation bit-width (A)=16, Quantization Protocol=None (Full Precision)2026.05 | 65.2 | — | — | — | — | — | — | — | — | |
| 3BASiLConfig=4:8+64LR, One-shot=true, Zero-shot=true2026.03 | 64.92 | 76.88 | 72.05 | 70.16 | 44.8 | 72.14 | 61.01 | 41.4 | 80.89 | |
| Hassle-free-ALPSConfig=4:8+64LR, One-shot=true, Zero-shot=true2026.03 | 63.91 | 76.82 | 71.19 | 71.04 | 44.45 | 72.77 | 56.68 | 40.2 | 78.13 | |
| 3BASiL-TMConfig=2:4+64LR, One-shot=true, Zero-shot=true2026.03 | 63.34 | 77.48 | 69.58 | 67.21 | 40.53 | 71.27 | 61.37 | 39.8 | 79.51 | |
| Hassle-free-SparseGPTConfig=4:8+64LR, One-shot=true, Zero-shot=true2026.03 | 63.22 | 77.09 | 69.95 | 69.32 | 41.81 | 71.27 | 56.32 | 40.6 | 79.39 | |
| 3BASiLConfig=2:4+64LR, One-shot=true, Zero-shot=true2026.03 | 62.8 | 76.5 | 67.61 | 67.21 | 40.1 | 70.24 | 64.26 | 38.2 | 78.29 | |
| OATSConfig=4:8+64LR, One-shot=true, Zero-shot=true2026.03 | 61.84 | 75.84 | 67.6 | 67.09 | 41.21 | 70.88 | 60.29 | 38.2 | 73.61 | |
| Hassle-free-ALPSConfig=2:4+64LR, One-shot=true, Zero-shot=true2026.03 | 61.61 | 75.19 | 67.1 | 64.44 | 38.91 | 69.53 | 59.93 | 39.4 | 78.38 | |
| Hassle-free-SparseGPTConfig=2:4+64LR, One-shot=true, Zero-shot=true2026.03 | 60.71 | 74.81 | 65.04 | 66.16 | 38.57 | 70.09 | 54.87 | 38.4 | 77.71 | |
| FP16 BaseBackbone=Qwen3-1.7B, Weight bit-width (W)=16, Activation bit-width (A)=16, Quantization Protocol=None (Full Precision)2026.05 | 58.1 | — | — | — | — | — | — | — | — | |
| 3BASiL-TMConfig=3:8+64LR, One-shot=true, Zero-shot=true2026.03 | 57.94 | 74.05 | 61.85 | 60.73 | 34.73 | 65.98 | 54.51 | 34.8 | 76.91 | |
| ParetoQ + JacQuantBackbone=LLaMA3-3B, Weight bit-width (W)=1.58, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 57.1 | — | — | — | — | — | — | — | — | |
| OATSConfig=2:4+64LR, One-shot=true, Zero-shot=true2026.03 | 56.99 | 72.74 | 60.7 | 60.86 | 34.81 | 65.51 | 57.76 | 35.2 | 68.32 | |
| ParetoQBackbone=LLaMA3-3B, Weight bit-width (W)=1.58, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 55.9 | — | — | — | — | — | — | — | — | |
| 3BASiLConfig=3:8+64LR, One-shot=true, Zero-shot=true2026.03 | 54.9 | 71.06 | 56.96 | 57.7 | 32.59 | 66.69 | 54.51 | 33 | 66.7 | |
| ParetoQ + JacQuantBackbone=LLaMA3-3B, Weight bit-width (W)=1, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 54.8 | — | — | — | — | — | — | — | — | |
| Hassle-free-ALPSConfig=3:8+64LR, One-shot=true, Zero-shot=true2026.03 | 54.55 | 70.62 | 56.54 | 54.42 | 30.12 | 64.72 | 55.23 | 32.8 | 71.96 | |
| ParetoQBackbone=LLaMA3-3B, Weight bit-width (W)=1, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 54 | — | — | — | — | — | — | — | — | |
| Hassle-free-SparseGPTConfig=3:8+64LR, One-shot=true, Zero-shot=true2026.03 | 52.78 | 68.66 | 51.99 | 50.97 | 30.38 | 63.85 | 53.07 | 32 | 71.31 | |
| ParetoQ + JacQuantBackbone=Qwen3-1.7B, Weight bit-width (W)=2, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 48 | — | — | — | — | — | — | — | — | |
| ParetoQBackbone=Qwen3-1.7B, Weight bit-width (W)=2, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 47.8 | — | — | — | — | — | — | — | — | |
| OATSConfig=3:8+64LR, One-shot=true, Zero-shot=true2026.03 | 46.15 | 63.71 | 39.48 | 42.68 | 24.32 | 53.91 | 52.71 | 28.4 | 63.98 | |
| ParetoQ + JacQuantBackbone=Qwen3-1.7B, Weight bit-width (W)=1, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 42.7 | — | — | — | — | — | — | — | — | |
| ParetoQBackbone=Qwen3-1.7B, Weight bit-width (W)=1, Activation bit-width (A)=8, Quantization Protocol=QAT2026.05 | 42.2 | — | — | — | — | — | — | — | — |