Commonsense Reasoning on HellaSwag (Robustness Evaluation)
82.94Accuracy (Baseline)BF16
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BF16Model=Qwen2.5-14B, A-W Quant Type=BF162026.02 | 82.94 | — | — | |
| NVFP4+PTSModel=Qwen2.5-14B, A-W Quant Type=NVFP4+PTS2026.02 | 81.45 | — | -1.49 | |
| HiF4Model=Qwen2.5-14B, A-W Quant Type=HiF42026.02 | 81.45 | — | -1.49 | |
| NVFP4Model=Qwen2.5-14B, A-W Quant Type=NVFP42026.02 | 81.38 | — | -1.56 | |
| HiF4+HiGPTQModel=Qwen2.5-14B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 81.38 | — | -1.56 | |
| BF16Model=Mistral-7B, A-W Quant Type=BF162026.02 | 80.5 | — | — | |
| NVFP4+PTSModel=Mistral-7B, A-W Quant Type=NVFP4+PTS2026.02 | 79.71 | — | -0.79 | |
| HiF4+HiGPTQModel=Mistral-7B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 79.48 | — | -1.02 | |
| HiF4Model=Mistral-7B, A-W Quant Type=HiF42026.02 | 79.41 | — | -1.09 | |
| BF16Model=Llama3-8B, A-W Quant Type=BF162026.02 | 79.15 | — | — | |
| HiF4Model=Llama3-8B, A-W Quant Type=HiF42026.02 | 77.84 | — | -1.31 | |
| NVFP4+PTSModel=Llama3-8B, A-W Quant Type=NVFP4+PTS2026.02 | 77.6 | — | -1.55 | |
| HiF4+HiGPTQModel=Llama3-8B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 77.47 | — | -1.68 | |
| NVFP4Model=Llama3-8B, A-W Quant Type=NVFP42026.02 | 76.56 | — | -2.59 | |
| BF16Model=Llama2-7B, A-W Quant Type=BF162026.02 | 75.99 | — | — | |
| NVFP4Model=Llama2-7B, A-W Quant Type=NVFP42026.02 | 75.02 | — | -0.97 | |
| HiF4Model=Llama2-7B, A-W Quant Type=HiF42026.02 | 75 | — | -0.99 | |
| NVFP4+PTSModel=Llama2-7B, A-W Quant Type=NVFP4+PTS2026.02 | 74.92 | — | -1.07 | |
| HiF4+HiGPTQModel=Llama2-7B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 74.91 | — | -1.08 | |
| Mamba2-2.7bBit-Flips=9, Attack=COBRA2025.12 | 62.69 | 1.33 | — | |
| Mamba-2.8bBit-Flips=1, Attack=COBRA2025.12 | 62.59 | 26 | — | |
| Quamba-2.8b-w8a8Bit-Flips=1, Attack=COBRA2025.12 | 62.06 | 0 | — | |
| Quamba-2.8b-w4a16Bit-Flips=1, Attack=COBRA2025.12 | 61.66 | 0 | — | |
| Mamba2-1.3bBit-Flips=5, Attack=COBRA2025.12 | 56.69 | 0 | — | |
| Mamba-1.4bBit-Flips=1, Attack=COBRA2025.12 | 55.76 | 0 | — | |
| Quamba-1.4b-w4a16Bit-Flips=1, Attack=COBRA2025.12 | 55.17 | 0 | — | |
| Hymba-1.5bBit-Flips=2, Attack=COBRA2025.12 | 53.55 | 0.67 | — | |
| Quamba-1.4b-w8a8Bit-Flips=4, Attack=COBRA2025.12 | 49.39 | 22.66 | — | |
| Mamba2-370mBit-Flips=4, Attack=COBRA2025.12 | 43.08 | 8 | — | |
| Mamba-370mBit-Flips=4, Attack=COBRA2025.12 | 43.04 | 22 | — | |
| NVFP4Model=Mistral-7B, A-W Quant Type=NVFP42026.02 | 26.69 | — | -53.81 |