Robustness Evaluation on MMLU (Accuracy Before/After Attack)
72.7MMLU Accuracy (Before Attack)GDF-BFA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GDF-BFABackbone=Qwen3-8B, # of bit flips=6, Quantization=INT82025.11 | 72.7 | 26.6 | |
| GDF-BFABackbone=Llama3-8B, # of bit flips=1, Quantization=INT82025.11 | 61.5 | 23.1 | |
| GDF-BFABackbone=Mistral-7B-v0.3, # of bit flips=1, Quantization=INT82025.11 | 58.9 | 24.12 | |
| Llama3-8BModel=Llama3-8B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | 57.3 | 22.9 | |
| GDF-BFABackbone=Llama2-13B, # of bit flips=3, Quantization=INT82025.11 | 52.07 | 24 | |
| Llama2-13BModel=Llama2-13B, # of bit flips=3, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | 50.05 | 22.9 | |
| GDF-BFABackbone=Llama2-7B, # of bit flips=1, Quantization=INT82025.11 | 41.41 | 22.9 | |
| Llama2-7BModel=Llama2-7B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | 39.51 | 25.5 |