Red-teaming Safety Evaluation on StrongReject
4ASRTAP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TAPTarget Model=Llama2-7b2026.04 | 4 | — | — | |
| PAIRTarget Model=Llama2-7b2026.04 | 5 | — | — | |
| PAP-top5Target Model=Gemma-7b2026.04 | 6 | — | — | |
| Meta-Llama-3.1-8B (ATTAQ-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=ATTAQ2025.05 | 7 | 11 | 1.79 | |
| Meta-Llama-3.1-8B (WILDBREAK-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=WILDBREAK2025.05 | 7 | 18 | 2.14 | |
| PAIRTarget Model=Gemma-7b2026.04 | 8 | — | — | |
| PAP-top5Target Model=Llama3-8b2026.04 | 8 | — | — | |
| Rainbow TeamingTarget Model=Llama2-7b2026.04 | 8 | — | — | |
| Rainbow TeamingTarget Model=Gemma-7b2026.04 | 8 | — | — | |
| Meta-Llama-3.1-8B (TRIDENT-EDGE-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=TRIDENT-EDGE2025.05 | 9 | 8 | 1.86 | |
| Rainbow TeamingTarget Model=Llama3-8b2026.04 | 9 | — | — | |
| Meta-Llama-3.1-8B (AART-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=AART2025.05 | 10 | 13 | 1.94 | |
| GCG-TTarget Model=Llama3-8b2026.04 | 10 | — | — | |
| GCG-TTarget Model=Gemma-7b2026.04 | 10 | — | — | |
| PAP-top5Target Model=Llama2-7b2026.04 | 10 | — | — | |
| GCG-TTarget Model=Llama2-7b2026.04 | 12 | — | — | |
| PAIRTarget Model=Llama3-8b2026.04 | 12 | — | — | |
| TAPTarget Model=Llama3-8b2026.04 | 13 | — | — | |
| TAPTarget Model=Gemma-7b2026.04 | 16 | — | — | |
| GCGTarget Model=Llama3.1-8B2026.01 | 18.27 | — | — | |
| Meta-Llama-3.1-8B (SAFE_RLHF-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=SAFE_RLHF2025.05 | 19 | 21 | 2.41 | |
| Meta-Llama-3.1-8B (WILDCHAT-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=WILDCHAT2025.05 | 20 | 23 | 2.48 | |
| AutoDANTarget Model=Llama2-7B2026.01 | 20.45 | — | — | |
| GCGTarget Model=Llama2-7B2026.01 | 20.77 | — | — | |
| Meta-Llama-3.1-8B (HH_RLHF-aligned)base_model=Meta-Llama-3.1-8B, alignment_dataset=HH_RLHF2025.05 | 23 | 33 | 2.47 | |
| Meta-Llama-3.1-8B (Unaligned)base_model=Meta-Llama-3.1-8B2025.05 | 25 | 69 | 2.66 | |
| GCGTarget Model=Qwen2.5-7B2026.01 | 34.82 | — | — | |
| STAR-TeamingAttacker Model=Llama3-8b, Target Model=Llama3-8b2026.04 | 43 | — | — | |
| STAR-TeamingAttacker Model=Gemma-7b, Target Model=Llama3-8b2026.04 | 45 | — | — | |
| STAR-TeamingAttacker Model=Llama3-8b, Target Model=Llama2-7b2026.04 | 50 | — | — | |
| STAR-TeamingAttacker Model=Gemma-7b, Target Model=Gemma-7b2026.04 | 55 | — | — | |
| DSNTarget Model=Qwen2.5-7B2026.01 | 56.87 | — | — | |
| STAR-TeamingAttacker Model=Gemma-7b, Target Model=Llama2-7b2026.04 | 57 | — | — | |
| AutoDANTarget Model=Llama3.1-8B2026.01 | 57.51 | — | — | |
| DSNTarget Model=Llama2-7B2026.01 | 59.11 | — | — | |
| DSNTarget Model=Llama3.1-8B2026.01 | 60.38 | — | — | |
| STAR-TeamingAttacker Model=Llama3-8b, Target Model=Gemma-7b2026.04 | 61 | — | — | |
| AdvPrefixTarget Model=Llama2-7B2026.01 | 65.18 | — | — | |
| ORTHOTarget Model=Llama2-7B2026.01 | 66.77 | — | — | |
| DSNTarget Model=Mistral-7B2026.01 | 71.57 | — | — | |
| AutoDANTarget Model=Mistral-7B2026.01 | 73.48 | — | — | |
| ORTHOTarget Model=Mistral-7B2026.01 | 74.44 | — | — | |
| AutoDANTarget Model=Qwen2.5-7B2026.01 | 75.08 | — | — | |
| AdvPrefixTarget Model=Llama3.1-8B2026.01 | 75.4 | — | — | |
| AdvPrefixTarget Model=Qwen2.5-7B2026.01 | 77.96 | — | — | |
| GCGTarget Model=Mistral-7B2026.01 | 82.43 | — | — | |
| ORTHOTarget Model=Llama3.1-8B2026.01 | 87.22 | — | — | |
| ORTHOTarget Model=Qwen2.5-7B2026.01 | 88.5 | — | — | |
| AdvPrefixTarget Model=Mistral-7B2026.01 | 90.1 | — | — | |
| GOSVTarget Model=Qwen2.5-7B2026.01 | 90.42 | — | — | |
| GOSVTarget Model=Llama2-7B2026.01 | 91.05 | — | — | |
| GOSVTarget Model=Llama3.1-8B2026.01 | 92.01 | — | — | |
| GOSVTarget Model=Mistral-7B2026.01 | 99.36 | — | — |