Prompt Injection Detection on AgentDojo (Precision/Recall Metrics)
92.1Source PrecisionPI-Hunter
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PI-HunterModel=Gemini-3.1-pro, Attack=important_inst2026.06 | 92.1 | 89.6 | 89.4 | 81.3 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=ignore_previous2026.06 | 92 | 91.5 | 92.2 | 87.1 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=system_msg2026.06 | 90.7 | 89.4 | 90.8 | 87.1 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=ignore_previous2026.06 | 89.1 | 81.6 | 93.8 | 77.8 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=direct2026.06 | 87.7 | 71.4 | 95.7 | 82.5 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=system_msg2026.06 | 87.7 | 70.9 | 92.8 | 69.3 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=direct2026.06 | 87.3 | 88.1 | 86.8 | 84 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=important_inst2026.06 | 87 | 64.3 | 90.5 | 74.2 | |
| BaselineModel=Claude-4.6-sonnet, Attack=direct2026.06 | 85.4 | 61.2 | 91.7 | 72.9 | |
| BaselineModel=Claude-4.6-sonnet, Attack=important_inst2026.06 | 83.9 | 59.2 | 84.4 | 67.7 | |
| PI-HunterModel=GPT-5.4-mini, Attack=direct2026.06 | 83.4 | 60.4 | 78.8 | 74.5 | |
| BaselineModel=Claude-4.6-sonnet, Attack=ignore_previous2026.06 | 82.9 | 57.9 | 86.8 | 63.5 | |
| BaselineModel=Claude-4.6-sonnet, Attack=system_msg2026.06 | 82.6 | 51.9 | 85.4 | 64.6 | |
| PI-HunterModel=GPT-5.4-mini, Attack=ignore_previous2026.06 | 81.2 | 77.1 | 82 | 82.6 | |
| PI-HunterModel=GPT-5.4-mini, Attack=agentvigil2026.06 | 80.3 | 67.1 | 75.3 | 65 | |
| PI-HunterModel=Gemini-3.1-pro, Attack=agentvigil2026.06 | 79.6 | 83.4 | 80.6 | 82.4 | |
| BaselineModel=GPT-5.4-mini, Attack=direct2026.06 | 77.3 | 49.7 | 71.7 | 57.3 | |
| PI-HunterModel=Claude-4.6-sonnet, Attack=agentvigil2026.06 | 77.3 | 68.1 | 80.7 | 57 | |
| PI-HunterModel=GPT-5.4-mini, Attack=system_msg2026.06 | 75.4 | 67.6 | 80 | 84.5 | |
| PI-HunterModel=GPT-5.4-mini, Attack=important_inst2026.06 | 73.4 | 61.9 | 70.7 | 84.8 | |
| BaselineModel=Claude-4.6-sonnet, Attack=agentvigil2026.06 | 73.4 | 51.6 | 72.5 | 49.6 | |
| BaselineModel=GPT-5.4-mini, Attack=ignore_previous2026.06 | 71.9 | 48.2 | 73.1 | 60.4 | |
| BaselineModel=GPT-5.4-mini, Attack=agentvigil2026.06 | 70.3 | 55.5 | 68.8 | 57.2 | |
| BaselineModel=GPT-5.4-mini, Attack=system_msg2026.06 | 59.4 | 46.4 | 56.9 | 44.8 | |
| BaselineModel=GPT-5.4-mini, Attack=important_inst2026.06 | 59 | 36.5 | 55.8 | 43.7 | |
| BaselineModel=Gemini-3.1-pro, Attack=system_msg2026.06 | 57.6 | 26.7 | 72.4 | 42.7 | |
| BaselineModel=Gemini-3.1-pro, Attack=agentvigil2026.06 | 55.8 | 25.5 | 67.5 | 43.6 | |
| BaselineModel=Gemini-3.1-pro, Attack=important_inst2026.06 | 50.5 | 24.1 | 68.8 | 37.5 | |
| BaselineModel=Gemini-3.1-pro, Attack=direct2026.06 | 49 | 24.8 | 60.9 | 42.2 | |
| BaselineModel=Gemini-3.1-pro, Attack=ignore_previous2026.06 | 36 | 22.1 | 50 | 34.8 |