Question Answering on ARC Challenge (Adversarial Robustness)
97.95Attack Success Rate (ASR)CacheTrap
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CacheTrapModel=LLaMA-3.1-8B, Target Class=02025.11 | 97.95 | — | 80.46 | |
| CacheTrapModel=Qwen-2.5-3B, Target Class=32025.11 | 99.57 | — | — | |
| CacheTrapModel=LLaMA-2-7B, Target Class=02025.11 | 99.91 | — | 65.1 | |
| CacheTrapModel=LLaMA-2-7B, Target Class=12025.11 | 100 | — | — | |
| CacheTrapModel=LLaMA-2-7B, Target Class=22025.11 | 100 | — | — | |
| CacheTrapModel=LLaMA-2-7B, Target Class=32025.11 | 100 | — | — | |
| CacheTrapModel=LLaMA-3.1-8B, Target Class=12025.11 | 100 | — | — | |
| CacheTrapModel=LLaMA-3.1-8B, Target Class=22025.11 | 100 | — | — | |
| CacheTrapModel=LLaMA-3.1-8B, Target Class=32025.11 | 100 | — | — | |
| CacheTrapModel=Mistral-7B, Target Class=02025.11 | 100 | — | 77.39 | |
| CacheTrapModel=Mistral-7B, Target Class=12025.11 | 100 | — | — | |
| CacheTrapModel=Mistral-7B, Target Class=22025.11 | 100 | — | — | |
| CacheTrapModel=Mistral-7B, Target Class=32025.11 | 100 | — | — | |
| CacheTrapModel=Qwen-2.5-3B, Target Class=02025.11 | 100 | — | 79.69 | |
| CacheTrapModel=Qwen-2.5-3B, Target Class=12025.11 | 100 | — | — | |
| CacheTrapModel=Qwen-2.5-3B, Target Class=22025.11 | 100 | — | — | |
| CacheTrapModel=DeepSeek-7B, Target Class=02025.11 | 100 | — | 71.33 | |
| CacheTrapModel=DeepSeek-7B, Target Class=12025.11 | 100 | — | — | |
| CacheTrapModel=DeepSeek-7B, Target Class=22025.11 | 100 | — | — | |
| CacheTrapModel=DeepSeek-7B, Target Class=32025.11 | 100 | — | — | |
| GDF-BFABackbone=Llama2-7B, # of bit flips=1, Quantization=INT82025.11 | — | 43.51 | 19.9 | |
| GDF-BFABackbone=Llama2-13B, # of bit flips=3, Quantization=INT82025.11 | — | 48.46 | 21.5 | |
| GDF-BFABackbone=Llama3-8B, # of bit flips=1, Quantization=INT82025.11 | — | 50.51 | 20.8 | |
| GDF-BFABackbone=Mistral-7B-v0.3, # of bit flips=1, Quantization=INT82025.11 | — | 48.8 | 21.5 | |
| GDF-BFABackbone=Qwen3-8B, # of bit flips=6, Quantization=INT82025.11 | — | 55.11 | 22.78 | |
| Llama2-13BModel=Llama2-13B, # of bit flips=3, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | — | 46.92 | 21 | |
| Llama2-7BModel=Llama2-7B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | — | 42.66 | 21.2 | |
| Llama3-8BModel=Llama3-8B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11 | — | 45.8 | 22.01 |