Question Answering on PIQA (Attack Robustness Evaluation)
77.31Accuracy (Baseline)Hymba-1.5b
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Hymba-1.5bBit-Flips=2, Attack=COBRA2025.12 | 77.31 | 2.67 | |
| Mamba2-2.7bBit-Flips=9, Attack=COBRA2025.12 | 74.86 | 17.33 | |
| Mamba-2.8bBit-Flips=1, Attack=COBRA2025.12 | 74.48 | 52 | |
| Quamba-2.8b-w8a8Bit-Flips=1, Attack=COBRA2025.12 | 73.99 | 0 | |
| Mamba-1.4bBit-Flips=1, Attack=COBRA2025.12 | 73.34 | 18 | |
| Quamba-2.8b-w4a16Bit-Flips=1, Attack=COBRA2025.12 | 73.29 | 25.33 | |
| Quamba-1.4b-w4a16Bit-Flips=1, Attack=COBRA2025.12 | 72.47 | 6 | |
| Mamba2-1.3bBit-Flips=5, Attack=COBRA2025.12 | 72.25 | 0 | |
| Quamba-1.4b-w8a8Bit-Flips=4, Attack=COBRA2025.12 | 71.76 | 49.33 | |
| Mamba2-370mBit-Flips=4, Attack=COBRA2025.12 | 68.77 | 24 | |
| Mamba-370mBit-Flips=4, Attack=COBRA2025.12 | 67.03 | 44.66 |