Commonsense Reasoning on Winogrande (Adversarial Robustness)
73.2Accuracy (Pre-Attack)Llama3-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama3-8BModel=Llama3-8B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | 73.2 | 48.2 | |
| GDF-BFABackbone=Llama3-8B, # of bit flips=1, Quantization=INT82025.11 | 73.16 | 49.8 | |
| GDF-BFABackbone=Mistral-7B-v0.3, # of bit flips=1, Quantization=INT82025.11 | 73 | 48.93 | |
| GDF-BFABackbone=Llama2-13B, # of bit flips=3, Quantization=INT82025.11 | 72.3 | 50.4 | |
| Llama2-13BModel=Llama2-13B, # of bit flips=3, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | 71.1 | 55.3 | |
| Llama2-7BModel=Llama2-7B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | 68.58 | 48.1 | |
| GDF-BFABackbone=Llama2-7B, # of bit flips=1, Quantization=INT82025.11 | 68.3 | 47.8 | |
| GDF-BFABackbone=Qwen3-8B, # of bit flips=6, Quantization=INT82025.11 | 68.2 | 53.9 | |
| Hymba-1.5bBit-Flips=2, Attack=COBRA2025.12 | 66.61 | 10.67 | |
| Quamba-2.8b-w4a16Bit-Flips=1, Attack=COBRA2025.12 | 58.01 | 46 | |
| Mamba-2.8bBit-Flips=1, Attack=COBRA2025.12 | 56.99 | 44.67 | |
| Quamba-2.8b-w8a8Bit-Flips=1, Attack=COBRA2025.12 | 56.59 | 0 | |
| Mamba2-2.7bBit-Flips=9, Attack=COBRA2025.12 | 56.59 | 5.33 | |
| Mamba-1.4bBit-Flips=1, Attack=COBRA2025.12 | 54.54 | 46 | |
| Quamba-1.4b-w4a16Bit-Flips=1, Attack=COBRA2025.12 | 53.57 | 6.67 | |
| Quamba-1.4b-w8a8Bit-Flips=4, Attack=COBRA2025.12 | 53.51 | 27.33 | |
| Mamba-370mBit-Flips=4, Attack=COBRA2025.12 | 52.72 | 46 | |
| Mamba2-370mBit-Flips=4, Attack=COBRA2025.12 | 51.46 | 7.33 | |
| Mamba2-1.3bBit-Flips=5, Attack=COBRA2025.12 | 49.33 | 4.67 |