LLM Attack Effectiveness Evaluation on LLM Attack Effectiveness Dataset
100Exact Match (EM)Kimi-k2.5
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Kimi-k2.5SC=NO, ST=COT, Defense Strategy=no defense2026.04 | 100 | 71.77 | 94.53 | 95 | |
| GPT-5SC=NO, ST=NO, Defense Strategy=no defense2026.04 | 90 | 98.66 | 99.85 | 99 | |
| Deepseek-V3.2SC=NO, ST=COT, Defense Strategy=no defense2026.04 | 80 | 67.53 | 91.83 | 93 | |
| Kimi-k2.5SC=NO, ST=NO, Defense Strategy=no defense2026.04 | 80 | 99.02 | 99.89 | 99 | |
| Deepseek-V3.2SC=ED, ST=FS, Defense Strategy=no defense2026.04 | 70 | 92.45 | 99.01 | 96 | |
| Deepseek-V3.2SC=ED, ST=NO, Defense Strategy=no defense2026.04 | 60 | 38.75 | 83.44 | 76 | |
| Kimi-k2.5SC=NO, ST=FS, Defense Strategy=no defense2026.04 | 60 | 99 | 99.89 | 99 | |
| AverageSC=RP, ST=COT, Defense Strategy=no defense2026.04 | 54 | 56.36 | 85.07 | 76 | |
| Deepseek-V3.2SC=HY, ST=COT, Defense Strategy=no defense2026.04 | 50 | 53.79 | 87.93 | 79 | |
| AverageSC=NO, ST=NO, Defense Strategy=no defense2026.04 | 48 | 79.95 | 95.11 | 91 | |
| MiniMax-M2.7SC=NO, ST=NO, Defense Strategy=no defense2026.04 | 40 | 88.21 | 97.65 | 95 | |
| Claude-haiku-4.5SC=NO, ST=NO, Defense Strategy=no defense2026.04 | 30 | 86 | 96.82 | 95 | |
| Deepseek-V3.2SC=ED, ST=COT, Defense Strategy=no defense2026.04 | 20 | 42.04 | 84.13 | 74 | |
| Deepseek-V3.2SC=HY, ST=NO, Defense Strategy=no defense2026.04 | 20 | 33.19 | 80.7 | 71 | |
| Deepseek-V3.2SC=HY, ST=FS, Defense Strategy=no defense2026.04 | 20 | 93.86 | 99.19 | 97 | |
| Deepseek-V3.2SC=RP, ST=NO, Defense Strategy=no defense2026.04 | 20 | 41.6 | 83.06 | 77 | |
| Deepseek-V3.2SC=NO, ST=FS, Defense Strategy=no defense2026.04 | 10 | 84.78 | 94.15 | 88 | |
| Deepseek-V3.2SC=RP, ST=FS, Defense Strategy=no defense2026.04 | 10 | 93.28 | 99.14 | 96 | |
| Deepseek-V3.2SC=RP, ST=COT, Defense Strategy=no defense2026.04 | 10 | 42.66 | 83.93 | 73 | |
| Deepseek-V3.2SC=NO, ST=NO, Defense Strategy=no defense2026.04 | 0 | 27.84 | 81.36 | 69 |