Prompt Injection Attack on InjecAgent
0ASR @ 1 AttemptDirect
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DirectTarget Model=Meta-SecAlign-8B, Attack Category=Static Attacks2026.03 | 0 | — | |
| DirectTarget Model=GPT-5-nano, Attack Category=Static Attacks2026.03 | 0 | — | |
| EnhancedTarget Model=GPT-5-nano, Attack Category=Static Attacks2026.03 | 0 | — | |
| EnhancedTarget Model=Meta-SecAlign-8B, Attack Category=Static Attacks2026.03 | 1 | — | |
| PAIRTarget Model=Meta-SecAlign-8B, Attack Category=Search-Based Attacks2026.03 | 1 | — | |
| DirectTarget Model=GPT-4.1-nano, Attack Category=Static Attacks2026.03 | 1 | — | |
| PAIRTarget Model=GPT-5-nano, Attack Category=Search-Based Attacks2026.03 | 1 | — | |
| DirectTarget Model=GPT-4o-mini, Attack Category=Static Attacks2026.03 | 2 | — | |
| EnhancedTarget Model=GPT-4.1-nano, Attack Category=Static Attacks2026.03 | 2 | — | |
| TAPTarget Model=Meta-SecAlign-8B, Attack Category=Search-Based Attacks2026.03 | 3 | — | |
| EnhancedTarget Model=GPT-4o-mini, Attack Category=Static Attacks2026.03 | 3 | — | |
| StrategyTarget Model=Meta-SecAlign-8B, Attack Category=Search-Based Attacks2026.03 | 4 | — | |
| Vanilla GRPOTarget Model=Meta-SecAlign-8B, Attack Category=RL-Based Attacks2026.03 | 6 | 11 | |
| TAPTarget Model=GPT-5-nano, Attack Category=Search-Based Attacks2026.03 | 8 | — | |
| Vanilla GRPOTarget Model=GPT-5-nano, Attack Category=RL-Based Attacks2026.03 | 11 | 24 | |
| StrategyTarget Model=GPT-5-nano, Attack Category=Search-Based Attacks2026.03 | 18 | — | |
| PAIRTarget Model=GPT-4o-mini, Attack Category=Search-Based Attacks2026.03 | 24 | — | |
| PAIRTarget Model=GPT-4.1-nano, Attack Category=Search-Based Attacks2026.03 | 32 | — | |
| StrategyTarget Model=GPT-4o-mini, Attack Category=Search-Based Attacks2026.03 | 38 | — | |
| TAPTarget Model=GPT-4o-mini, Attack Category=Search-Based Attacks2026.03 | 40 | — | |
| Vanilla GRPOTarget Model=GPT-4o-mini, Attack Category=RL-Based Attacks2026.03 | 46 | 60 | |
| TAPTarget Model=GPT-4.1-nano, Attack Category=Search-Based Attacks2026.03 | 54 | — | |
| Vanilla GRPOTarget Model=GPT-4.1-nano, Attack Category=RL-Based Attacks2026.03 | 61 | 75 | |
| StrategyTarget Model=GPT-4.1-nano, Attack Category=Search-Based Attacks2026.03 | 65 | — | |
| RL-HammerTarget Model=GPT-5-nano, Attack Category=RL-Based Attacks2026.03 | 86 | 96 | |
| RL-HammerTarget Model=GPT-4o-mini, Attack Category=RL-Based Attacks2026.03 | 94 | 100 | |
| RL-HammerTarget Model=GPT-4.1-nano, Attack Category=RL-Based Attacks2026.03 | 95 | 100 | |
| PISmithTarget Model=GPT-5-nano, Attack Category=RL-Based Attacks2026.03 | 95 | 100 | |
| RL-HammerTarget Model=Meta-SecAlign-8B, Attack Category=RL-Based Attacks2026.03 | 98 | 100 | |
| PISmithTarget Model=Meta-SecAlign-8B, Attack Category=RL-Based Attacks2026.03 | 99 | 100 | |
| PISmithTarget Model=GPT-4o-mini, Attack Category=RL-Based Attacks2026.03 | 99 | 100 | |
| PISmithTarget Model=GPT-4.1-nano, Attack Category=RL-Based Attacks2026.03 | 100 | 100 |