List your company
ResearchFormerly WizWand

Question Answering on ARC Challenge (Adversarial Robustness)

97.95Attack Success Rate (ASR)CacheTrap
97.86898.421598.97599.5285Nov 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
CacheTrapModel=LLaMA-3.1-8B, Target Class=02025.1197.95—80.46
CacheTrapModel=Qwen-2.5-3B, Target Class=32025.1199.57——
CacheTrapModel=LLaMA-2-7B, Target Class=02025.1199.91—65.1
CacheTrapModel=LLaMA-2-7B, Target Class=12025.11100——
CacheTrapModel=LLaMA-2-7B, Target Class=22025.11100——
CacheTrapModel=LLaMA-2-7B, Target Class=32025.11100——
CacheTrapModel=LLaMA-3.1-8B, Target Class=12025.11100——
CacheTrapModel=LLaMA-3.1-8B, Target Class=22025.11100——
CacheTrapModel=LLaMA-3.1-8B, Target Class=32025.11100——
CacheTrapModel=Mistral-7B, Target Class=02025.11100—77.39
CacheTrapModel=Mistral-7B, Target Class=12025.11100——
CacheTrapModel=Mistral-7B, Target Class=22025.11100——
CacheTrapModel=Mistral-7B, Target Class=32025.11100——
CacheTrapModel=Qwen-2.5-3B, Target Class=02025.11100—79.69
CacheTrapModel=Qwen-2.5-3B, Target Class=12025.11100——
CacheTrapModel=Qwen-2.5-3B, Target Class=22025.11100——
CacheTrapModel=DeepSeek-7B, Target Class=02025.11100—71.33
CacheTrapModel=DeepSeek-7B, Target Class=12025.11100——
CacheTrapModel=DeepSeek-7B, Target Class=22025.11100——
CacheTrapModel=DeepSeek-7B, Target Class=32025.11100——
GDF-BFABackbone=Llama2-7B, # of bit flips=1, Quantization=INT82025.11—43.5119.9
GDF-BFABackbone=Llama2-13B, # of bit flips=3, Quantization=INT82025.11—48.4621.5
GDF-BFABackbone=Llama3-8B, # of bit flips=1, Quantization=INT82025.11—50.5120.8
GDF-BFABackbone=Mistral-7B-v0.3, # of bit flips=1, Quantization=INT82025.11—48.821.5
GDF-BFABackbone=Qwen3-8B, # of bit flips=6, Quantization=INT82025.11—55.1122.78
Llama2-13BModel=Llama2-13B, # of bit flips=3, Quantization precision=4-bit, Attack=GDF-BFA2025.11—46.9221
Llama2-7BModel=Llama2-7B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11—42.6621.2
Llama3-8BModel=Llama3-8B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.11—45.822.01