Jailbreaking on EHRAgent MIMIC-III
56.55SRNon-attack
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Non-attackAgent Backbone=GPT-52026.04 | 56.55 | — | — | — | 87.76 | |
| Non-attackAgent Backbone=GPT-4o2026.04 | 55.86 | — | — | — | 84.31 | |
| JailAgentAgent Backbone=GPT-4o2026.04 | 55.52 | 65.17 | 59.48 | 56.21 | 83.62 | |
| JailAgentAgent Backbone=GPT-52026.04 | 55.52 | 66.38 | 60.17 | 56.72 | 86.72 | |
| AgentPoisonAgent Backbone=GPT-52026.04 | 52.07 | 62.41 | 58.62 | 54.14 | 87.93 | |
| AgentPoisonAgent Backbone=GPT-4o2026.04 | 51.9 | 63.28 | 51.21 | 51.03 | 80.34 | |
| BadChainAgent Backbone=GPT-52026.04 | 51.38 | 62.07 | 58.97 | 57.59 | 86.03 | |
| BadChainAgent Backbone=GPT-4o2026.04 | 47.76 | 62.76 | 49.66 | 48.1 | 78.62 | |
| PAIRAgent Backbone=GPT-52026.04 | 47.76 | 51.9 | 47.24 | 44.48 | 84.14 | |
| JailAgentAgent Backbone=Gemini-3.0-pro2026.04 | 47.41 | 48.97 | 44.83 | 45.34 | 88.45 | |
| Non-attackAgent Backbone=GPT-3.5-turbo2026.04 | 46.72 | — | — | — | 56.9 | |
| Non-attackAgent Backbone=Gemini-3.0-pro2026.04 | 46.72 | — | — | — | 89.48 | |
| AgentPoisonAgent Backbone=Claude-3.5-haiku2026.04 | 46.38 | 70.52 | 66.9 | 65.52 | 97.93 | |
| Non-attackAgent Backbone=Claude-3.5-haiku2026.04 | 46.03 | — | — | — | 98.62 | |
| JailAgentAgent Backbone=GPT-3.5-turbo2026.04 | 45.86 | 64.31 | 59.31 | 55.86 | 56.03 | |
| JailAgentAgent Backbone=Claude-3.5-haiku2026.04 | 45.17 | 70.34 | 67.07 | 65 | 97.59 | |
| BadChainAgent Backbone=Claude-3.5-haiku2026.04 | 44.14 | 67.41 | 65.17 | 62.76 | 96.21 | |
| AgentPoisonAgent Backbone=Gemini-3.0-pro2026.04 | 43.28 | 45.34 | 42.41 | 41.55 | 86.03 | |
| JailAgentAgent Backbone=Llama-3.1-70B2026.04 | 42.93 | 63.97 | 61.21 | 58.97 | 81.9 | |
| Non-attackAgent Backbone=Llama-3.1-70B2026.04 | 42.41 | — | — | — | 81.38 | |
| PAIRAgent Backbone=Claude-3.5-haiku2026.04 | 42.07 | 57.93 | 51.72 | 49.48 | 94.31 | |
| BadChainAgent Backbone=Gemini-3.0-pro2026.04 | 42.07 | 43.97 | 40.52 | 40.17 | 82.07 | |
| BadChainAgent Backbone=Llama-3.1-70B2026.04 | 41.72 | 62.41 | 57.07 | 56.21 | 78.62 | |
| AgentPoisonAgent Backbone=Llama-3.1-70B2026.04 | 41.55 | 61.9 | 56.38 | 55.86 | 78.45 | |
| PAIRAgent Backbone=GPT-4o2026.04 | 40.52 | 45 | 38.28 | 38.28 | 73.28 | |
| BadChainAgent Backbone=GPT-3.5-turbo2026.04 | 39.83 | 61.03 | 54.31 | 51.21 | 43.97 | |
| PAIRAgent Backbone=Gemini-3.0-pro2026.04 | 39.83 | 36.03 | 34.66 | 34.31 | 78.62 | |
| AgentPoisonAgent Backbone=GPT-3.5-turbo2026.04 | 38.28 | 58.79 | 51.38 | 49.48 | 42.41 | |
| PAIRAgent Backbone=GPT-3.5-turbo2026.04 | 35.86 | 42.24 | 37.07 | 36.55 | 39.31 | |
| PAIRAgent Backbone=Llama-3.1-70B2026.04 | 33.97 | 52.41 | 48.1 | 47.76 | 70.17 |