Prompt Injection Detection on AgentDyn
83.1Source PrecisionPI-Hunter
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PI-HunterModel=Gemini-3.1-pro, Attack=direct2026.06 | 83.1 | 87.1 | 91 | 86.1 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=system_msg2026.06 | 82.8 | 86 | 87.5 | 84.3 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=important_inst2026.06 | 80.8 | 82.9 | 83.3 | 80.3 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=ignore_previous2026.06 | 80.5 | 82.8 | 85 | 82.2 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=agentvigil2026.06 | 76.2 | 75.5 | 82 | 77.5 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=ignore_previous2026.06 | 74.7 | 88.9 | 83.1 | 74.8 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=agentvigil2026.06 | 73.3 | 79.4 | 73.2 | 66.4 | |
| PI-HunterModel=GPT-5.4-mini, Attack=system_msg2026.06 | 71.1 | 73.3 | 70.7 | 74.1 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=direct2026.06 | 70.5 | 73.2 | 75.9 | 79.2 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=system_msg2026.06 | 69.8 | 82.5 | 75.5 | 71.4 | |
| PI-HunterModel=GPT-5.4-mini, Attack=ignore_previous2026.06 | 69.1 | 74.2 | 68.2 | 81.6 | |
| PI-HunterModel=GPT-5.4-mini, Attack=direct2026.06 | 68.8 | 68.8 | 68.8 | 79.3 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=important_inst2026.06 | 68.2 | 85.7 | 75.3 | 73 | |
| BaselineModel=Gemini-3.1-pro, Attack=agentvigil2026.06 | 61.8 | 41.2 | 64.1 | 28.8 | |
| BaselineModel=Claude-4.6-sonnet, Attack=ignore_previous2026.06 | 61.4 | 35.6 | 71.5 | 48.4 | |
| BaselineModel=GPT-5.4-mini, Attack=ignore_previous2026.06 | 61.3 | 36.3 | 56.5 | 44.5 | |
| BaselineModel=GPT-5.4-mini, Attack=system_msg2026.06 | 61 | 36.3 | 58.7 | 34.8 | |
| PI-HunterModel=GPT-5.4-mini, Attack=agentvigil2026.06 | 60.5 | 75.6 | 70.4 | 78.6 | |
| BaselineModel=Gemini-3.1-pro, Attack=important_inst2026.06 | 57.6 | 22.7 | 71.2 | 40.5 | |
| BaselineModel=Claude-4.6-sonnet, Attack=important_inst2026.06 | 57.2 | 32.8 | 69.4 | 46.2 | |
| PI-HunterModel=GPT-5.4-mini, Attack=important_inst2026.06 | 56.2 | 76.5 | 64 | 72.3 | |
| BaselineModel=Claude-4.6-sonnet, Attack=direct2026.06 | 54.7 | 37.8 | 47.6 | 44.7 | |
| BaselineModel=Gemini-3.1-pro, Attack=system_msg2026.06 | 50.7 | 36.3 | 51.8 | 54.4 | |
| BaselineModel=Claude-4.6-sonnet, Attack=system_msg2026.06 | 49.6 | 28.1 | 57.5 | 45.8 | |
| BaselineModel=Gemini-3.1-pro, Attack=ignore_previous2026.06 | 47.2 | 26.2 | 63.4 | 33 | |
| BaselineModel=Gemini-3.1-pro, Attack=direct2026.06 | 44.4 | 27.1 | 65.3 | 38.1 | |
| BaselineModel=Claude-4.6-sonnet, Attack=agentvigil2026.06 | 40.3 | 30 | 48.5 | 37.8 | |
| BaselineModel=GPT-5.4-mini, Attack=important_inst2026.06 | 38.9 | 24.3 | 36.8 | 21.9 | |
| BaselineModel=GPT-5.4-mini, Attack=agentvigil2026.06 | 38.7 | 25.3 | 38.5 | 22.4 | |
| BaselineModel=GPT-5.4-mini, Attack=direct2026.06 | 36 | 31.4 | 34 | 45.8 |