List your company
ResearchFormerly WizWand

Robustness Evaluation: Question Answering on ARC Easy

93.01Accuracy (After Attack)CacheTrap
-3.720421.392346.50571.6177Nov 27, 2025Nov 29, 2025Dec 2, 2025Dec 5, 2025Dec 8, 2025Dec 11, 2025Dec 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
CacheTrapModel=Qwen-2.5-3B, Target Class=02025.1193.01—100
CacheTrapModel=Qwen-2.5-3B, Target Class=12025.1193.01—100
CacheTrapModel=Qwen-2.5-3B, Target Class=22025.1193.01—100
CacheTrapModel=Qwen-2.5-3B, Target Class=32025.1193.01—100
CacheTrapModel=Qwen-2.5-3B, Target Class=42025.1193.01—100
CacheTrapModel=LLaMA-3.1-8B, Target Class=02025.1192.83—100
CacheTrapModel=LLaMA-3.1-8B, Target Class=12025.1192.83—100
CacheTrapModel=LLaMA-3.1-8B, Target Class=22025.1192.83—100
CacheTrapModel=LLaMA-3.1-8B, Target Class=32025.1192.83—100
CacheTrapModel=LLaMA-3.1-8B, Target Class=42025.1192.83—100
CacheTrapModel=Mistral-7B, Target Class=02025.1189.69—100
CacheTrapModel=Mistral-7B, Target Class=12025.1189.69—100
CacheTrapModel=Mistral-7B, Target Class=22025.1189.69—100
CacheTrapModel=Mistral-7B, Target Class=32025.1189.69—100
CacheTrapModel=Mistral-7B, Target Class=42025.1189.69—100
CacheTrapModel=DeepSeek-7B, Target Class=02025.1186.78—100
CacheTrapModel=DeepSeek-7B, Target Class=12025.1186.78—100
CacheTrapModel=DeepSeek-7B, Target Class=22025.1186.78—100
CacheTrapModel=DeepSeek-7B, Target Class=32025.1186.78—100
CacheTrapModel=DeepSeek-7B, Target Class=42025.1186.78—100
CacheTrapModel=LLaMA-2-7B, Target Class=02025.1183.46—100
CacheTrapModel=LLaMA-2-7B, Target Class=12025.1183.46—100
CacheTrapModel=LLaMA-2-7B, Target Class=22025.1183.46—100
CacheTrapModel=LLaMA-2-7B, Target Class=32025.1183.46—100
CacheTrapModel=LLaMA-2-7B, Target Class=42025.1183.46—100
Quamba-2.8b-w4a16Bit-Flips=1, Attack=COBRA2025.126068.7—
GDF-BFABackbone=Qwen3-8B, # of bit flips=6, Quantization=INT82025.1138.282.87—
Mamba-370mBit-Flips=4, Attack=COBRA2025.1228.6753.125—
GDF-BFABackbone=Mistral-7B-v0.3, # of bit flips=1, Quantization=INT82025.1128.679.62—
Llama2-13BModel=Llama2-13B, # of bit flips=3, Quantization precision=4-bit, Attack=GDF-BFA2025.1127.877.56—
GDF-BFABackbone=Llama2-7B, # of bit flips=1, Quantization=INT82025.1127.1875.79—
Mamba-2.8bBit-Flips=1, Attack=COBRA2025.1226.6765.625—
GDF-BFABackbone=Llama3-8B, # of bit flips=1, Quantization=INT82025.1126.480.09—
GDF-BFABackbone=Llama2-13B, # of bit flips=3, Quantization=INT82025.1125.979.41—
Llama3-8BModel=Llama3-8B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.1125.1678.5—
Llama2-7BModel=Llama2-7B, # of bit flips=1, Quantization precision=4-bit, Attack=GDF-BFA2025.1124.875.2—
Quamba-1.4b-w8a8Bit-Flips=4, Attack=COBRA2025.122463.15—
Mamba-1.4bBit-Flips=1, Attack=COBRA2025.121462.5—
Mamba2-370mBit-Flips=4, Attack=COBRA2025.1212.6746.975—
Mamba2-2.7bBit-Flips=9, Attack=COBRA2025.12266.41—
Quamba-1.4b-w4a16Bit-Flips=1, Attack=COBRA2025.12061.75—
Mamba2-1.3bBit-Flips=5, Attack=COBRA2025.12062.5—
Quamba-2.8b-w8a8Bit-Flips=1, Attack=COBRA2025.12065.26—
Hymba-1.5bBit-Flips=2, Attack=COBRA2025.12076.94—