Jailbreaking on EHRAgent TREQS
71.78SRNon-attack
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Non-attackAgent Backbone=Gemini-3.0-pro2026.04 | 71.78 | — | — | — | 92.23 | |
| JailAgentAgent Backbone=Gemini-3.0-pro2026.04 | 70.86 | 63.91 | 62.47 | 61.25 | 93.05 | |
| BadChainAgent Backbone=Gemini-3.0-pro2026.04 | 66.16 | 54.5 | 51.33 | 50.92 | 88.85 | |
| AgentPoisonAgent Backbone=Gemini-3.0-pro2026.04 | 61.96 | 53.48 | 51.43 | 49.08 | 86.4 | |
| JailAgentAgent Backbone=GPT-4o2026.04 | 57.87 | 71.88 | 69.02 | 66.97 | 77.2 | |
| Non-attackAgent Backbone=GPT-4o2026.04 | 57.36 | — | — | — | 77.81 | |
| PAIRAgent Backbone=Gemini-3.0-pro2026.04 | 55.93 | 46.52 | 44.27 | 42.13 | 80.67 | |
| AgentPoisonAgent Backbone=GPT-4o2026.04 | 54.81 | 72.8 | 69.22 | 65.85 | 73.93 | |
| BadChainAgent Backbone=GPT-4o2026.04 | 53.17 | 65.03 | 61.35 | 60.12 | 71.68 | |
| JailAgentAgent Backbone=GPT-3.5-turbo2026.04 | 50.1 | 64.83 | 62.58 | 59.2 | 60.94 | |
| Non-attackAgent Backbone=GPT-52026.04 | 50 | — | — | — | 71.57 | |
| Non-attackAgent Backbone=GPT-3.5-turbo2026.04 | 49.9 | — | — | — | 60.63 | |
| PAIRAgent Backbone=GPT-4o2026.04 | 47.96 | 53.37 | 51.02 | 49.59 | 65.34 | |
| JailAgentAgent Backbone=GPT-52026.04 | 47.34 | 68.1 | 63.7 | 61.55 | 71.78 | |
| AgentPoisonAgent Backbone=GPT-52026.04 | 46.83 | 60.94 | 56.75 | 53.37 | 65.85 | |
| BadChainAgent Backbone=GPT-52026.04 | 45.4 | 64.52 | 62.47 | 60.94 | 63.7 | |
| BadChainAgent Backbone=GPT-3.5-turbo2026.04 | 44.58 | 61.76 | 59 | 53.8 | 54.81 | |
| Non-attackAgent Backbone=Llama-3.1-70B2026.04 | 44.58 | — | — | — | 82.31 | |
| JailAgentAgent Backbone=Llama-3.1-70B2026.04 | 43.46 | 52.04 | 46.42 | 45.91 | 82.11 | |
| AgentPoisonAgent Backbone=GPT-3.5-turbo2026.04 | 43.15 | 60.53 | 55.93 | 56.13 | 53.78 | |
| PAIRAgent Backbone=GPT-52026.04 | 42.94 | 55.11 | 56.03 | 49.9 | 61.55 | |
| AgentPoisonAgent Backbone=Llama-3.1-70B2026.04 | 41.41 | 51.02 | 46.52 | 45.4 | 79.75 | |
| PAIRAgent Backbone=GPT-3.5-turbo2026.04 | 40.59 | 58.18 | 54.6 | 51.23 | 51.33 | |
| Non-attackAgent Backbone=Claude-3.5-haiku2026.04 | 40.18 | — | — | — | 99.49 | |
| BadChainAgent Backbone=Llama-3.1-70B2026.04 | 39.78 | 47.96 | 44.27 | 42.02 | 76.58 | |
| JailAgentAgent Backbone=Claude-3.5-haiku2026.04 | 39.78 | 54.81 | 51.74 | 51.53 | 99.18 | |
| AgentPoisonAgent Backbone=Claude-3.5-haiku2026.04 | 38.34 | 49.9 | 46.63 | 44.58 | 98.06 | |
| PAIRAgent Backbone=Llama-3.1-70B2026.04 | 36.4 | 43.05 | 39.98 | 39.67 | 71.88 | |
| BadChainAgent Backbone=Claude-3.5-haiku2026.04 | 36.4 | 47.55 | 44.17 | 43.25 | 96.63 | |
| PAIRAgent Backbone=Claude-3.5-haiku2026.04 | 33.23 | 44.89 | 43.15 | 42.43 | 92.02 |