Jailbreaking on EHRAgent eICU
57.93Success Rate (SR)Non-attack
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Non-attackAgent Backbone=Claude-3.5-haiku2026.04 | 57.93 | — | — | — | 98.1 | |
| JailAgentAgent Backbone=Claude-3.5-haiku2026.04 | 57.24 | 95.69 | 91.21 | 90 | 97.76 | |
| BadChainAgent Backbone=Gemini-3.0-pro2026.04 | 51.72 | 80.34 | 78.62 | 76.55 | 79.66 | |
| AgentPoisonAgent Backbone=Claude-3.5-haiku2026.04 | 51.38 | 87.76 | 81.03 | 80.34 | 96.38 | |
| Non-attackAgent Backbone=Gemini-3.0-pro2026.04 | 50.86 | — | — | — | 80.34 | |
| JailAgentAgent Backbone=Gemini-3.0-pro2026.04 | 50.52 | 91.72 | 86.55 | 85.86 | 79.48 | |
| Non-attackAgent Backbone=GPT-4o2026.04 | 49.66 | — | — | — | 69.31 | |
| BadChainAgent Backbone=Claude-3.5-haiku2026.04 | 48.1 | 86.21 | 78.1 | 77.41 | 95.52 | |
| JailAgentAgent Backbone=GPT-4o2026.04 | 47.93 | 93.45 | 91.9 | 91.38 | 69.31 | |
| AgentPoisonAgent Backbone=Gemini-3.0-pro2026.04 | 47.76 | 80.86 | 77.41 | 75.69 | 76.9 | |
| PAIRAgent Backbone=Claude-3.5-haiku2026.04 | 47.24 | 73.97 | 70.52 | 67.07 | 93.79 | |
| JailAgentAgent Backbone=Llama-3.1-70B2026.04 | 46.55 | 82.59 | 78.62 | 76.72 | 81.03 | |
| Non-attackAgent Backbone=GPT-52026.04 | 46.38 | — | — | — | 63.28 | |
| Non-attackAgent Backbone=Llama-3.1-70B2026.04 | 46.38 | — | — | — | 80.17 | |
| AgentPoisonAgent Backbone=GPT-4o2026.04 | 46.21 | 87.76 | 85.52 | 82.59 | 62.07 | |
| AgentPoisonAgent Backbone=GPT-52026.04 | 45.69 | 86.21 | 79.14 | 77.41 | 63.28 | |
| BadChainAgent Backbone=GPT-4o2026.04 | 45.34 | 86.72 | 83.97 | 82.07 | 61.38 | |
| JailAgentAgent Backbone=GPT-52026.04 | 45.34 | 91.21 | 89.31 | 89.48 | 62.59 | |
| Non-attackAgent Backbone=GPT-3.5-turbo2026.04 | 44.48 | — | — | — | 61.72 | |
| JailAgentAgent Backbone=GPT-3.5-turbo2026.04 | 43.79 | 90.34 | 84.31 | 80.86 | 61.21 | |
| BadChainAgent Backbone=Llama-3.1-70B2026.04 | 43.28 | 80.52 | 75.69 | 75.52 | 79.31 | |
| AgentPoisonAgent Backbone=Llama-3.1-70B2026.04 | 42.93 | 75.86 | 71.21 | 70.69 | 78.79 | |
| PAIRAgent Backbone=GPT-4o2026.04 | 42.76 | 74.66 | 72.76 | 70.86 | 55.17 | |
| BadChainAgent Backbone=GPT-52026.04 | 42.24 | 85.86 | 76.55 | 74.66 | 57.93 | |
| PAIRAgent Backbone=Gemini-3.0-pro2026.04 | 42.24 | 70.86 | 68.97 | 65 | 67.59 | |
| AgentPoisonAgent Backbone=GPT-3.5-turbo2026.04 | 42.07 | 84.14 | 78.97 | 78.45 | 56.21 | |
| BadChainAgent Backbone=GPT-3.5-turbo2026.04 | 40.86 | 83.62 | 77.59 | 77.93 | 53.28 | |
| PAIRAgent Backbone=Llama-3.1-70B2026.04 | 39.48 | 66.72 | 61.9 | 58.45 | 77.93 | |
| PAIRAgent Backbone=GPT-52026.04 | 36.72 | 75.34 | 69.14 | 69.14 | 52.76 | |
| PAIRAgent Backbone=GPT-3.5-turbo2026.04 | 34.48 | 67.93 | 65 | 62.41 | 49.48 |