Commonsense Reasoning on HellaSwag (Accuracy, Delta)
0.061DeltaKICK-SETTLE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| KICK-SETTLEDefense=SDD, Model=Llama-3.1-8B, Attack Type=Kick-settle2026.05 | 0.061 | — | |
| KICK-SETTLEDefense=SDD, Model=Llama-2-7B, Attack Type=Kick-settle2026.05 | 0.041 | — | |
| KICK-SETTLEDefense=SDD, Model=Qwen3-8B, Attack Type=Kick-settle2026.05 | 0.005 | — | |
| FlashNormModel=Llama-3.1-8B, Few-shot setting=0-shot, Context window=2048-token, Precision=fp16, Hardware=NVIDIA A100, Evaluation framework=lm-evaluation-harness2024.07 | 0.0001 | 60.72 | |
| SourceModel=Llama-3.1-8B, Few-shot setting=0-shot, Context window=2048-token, Precision=fp16, Hardware=NVIDIA A100, Evaluation framework=lm-evaluation-harness2024.07 | — | 60.71 |