Commonsense Reasoning on PIQA (Accuracy and Normalized Accuracy)
85.47Normalized AccuracyN-3-Super 120B-A12B-Base
Evaluation results
| Method | Links | ||
|---|---|---|---|
| N-3-Super 120B-A12B-BaseShots=02026.04 | 85.47 | — | |
| GLM-4.5 Air-BaseShots=02026.04 | 84.22 | — | |
| Ling-flash base-2.0Shots=02026.04 | 84 | — | |
| SoftmaxQuantization=8-bit2025.04 | 73.83 | 73.72 | |
| SoftmaxQuantization=4-bit2025.04 | 73.34 | 71.98 | |
| SoftmaxQuantization=8-bit2025.04 | 72.96 | 73.56 | |
| SoftmaxQuantization=4-bit2025.04 | 72.14 | 72.63 | |
| SoftmaxQuantization=3-bit2025.04 | 71 | 69.91 | |
| SoftpickQuantization=8-bit2025.04 | 70.67 | 71.27 | |
| SoftpickQuantization=4-bit2025.04 | 70.57 | 71 | |
| SoftpickQuantization=3-bit2025.04 | 70.51 | 70.18 | |
| SoftpickQuantization=8-bit2025.04 | 70.4 | 71.22 | |
| SoftpickQuantization=4-bit2025.04 | 69.97 | 71.11 | |
| SoftmaxQuantization=8-bit2025.04 | 66.59 | 66.81 | |
| SoftpickQuantization=8-bit2025.04 | 66.54 | 66.21 | |
| SoftpickQuantization=4-bit2025.04 | 66.43 | 66.59 | |
| SoftpickQuantization=4-bit2025.04 | 66.38 | 66.32 | |
| SoftmaxQuantization=8-bit2025.04 | 66.32 | 67.36 | |
| SoftpickQuantization=8-bit2025.04 | 66.32 | 66.38 | |
| SoftmaxQuantization=4-bit2025.04 | 66.1 | 65.89 | |
| SoftpickQuantization=3-bit2025.04 | 65.78 | 66 | |
| SoftmaxQuantization=4-bit2025.04 | 65.29 | 65.4 | |
| SoftpickQuantization=3-bit2025.04 | 64.64 | 65.56 | |
| SoftmaxQuantization=3-bit2025.04 | 63.76 | 64.53 | |
| SoftmaxQuantization=3-bit2025.04 | 62.4 | 63.55 | |
| MoEScale=L, Size=808.4M, FLOPs=608.4M, Zero-shot=true2026.04 | 59.19 | — | |
| RFMoEScale=M, Size=307.3M, FLOPs=249.2M, Zero-shot=true2026.04 | 58.92 | — | |
| RFMoEScale=L, Size=870.6M, FLOPs=613.2M, Zero-shot=true2026.04 | 58.87 | — | |
| RFMoEScale=S, Size=95.32M, FLOPs=91.08M, Zero-shot=true2026.04 | 58.49 | — | |
| MoEScale=M, Size=289.9M, FLOPs=248.0M, Zero-shot=true2026.04 | 58.32 | — | |
| MoEScale=S, Size=92.44M, FLOPs=90.93M, Zero-shot=true2026.04 | 57.56 | — | |
| ReMoEScale=S, Size=92.44M, FLOPs=90.93M, Zero-shot=true2026.04 | 56.58 | — | |
| AoEScale=S, Size=93.85M, FLOPs=88.57M, Zero-shot=true2026.04 | 56.09 | — | |
| SoftpickQuantization=2-bit2025.04 | 55.11 | 56.04 | |
| SoftmaxQuantization=2-bit2025.04 | 53.97 | 54.46 | |
| SoftpickQuantization=2-bit2025.04 | 53.92 | 55.22 | |
| SoftpickQuantization=2-bit2025.04 | 53.59 | 54.52 | |
| SoftmaxQuantization=2-bit2025.04 | 50.38 | 52.29 | |
| SoftmaxQuantization=2-bit2025.04 | 48.42 | 52.12 | |
| Swimba-14BParameters=14B, Experts=4, FLOPs / token=1.51282 × 10^102026.03 | 0.825 | 0.82 | |
| Nemotron-H-8BParameters=8B, FLOPs / token=1.51263 × 10^102026.03 | 0.823 | 0.817 | |
| ASVDModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | — | 52.5 | |
| COALAμModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | — | 62.8 | |
| COALAμ=0Model=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | — | 60.9 | |
| OriginalModel=LLaMA-3.2-1B-Instruct, Compression Ratio=100%, Precision=fp162025.07 | — | 74.4 | |
| SVD-LLMModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | — | 60.6 |