Safety Compliance Evaluation on eICU-AC
98.4LPAAGrail
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AGrailDefense Category=Guardrail-based, Base Agent=Claude-3.52026.02 | 98.4 | 97 | 100 | 98.5 | 100 | |
| Spider-SenseDefense Category=Guardrail-based, Base Agent=Claude-3.52026.02 | 96.7 | 97.4 | 100 | 98.1 | 100 | |
| Spider-SenseDefense Category=Guardrail-based, Base Agent=Qwen-max2026.02 | 92.6 | 95.8 | 100 | 86.9 | 100 | |
| AGrailDefense Category=Guardrail-based, Base Agent=Qwen-max2026.02 | 90.6 | 85.3 | 93.5 | 67 | 100 | |
| GuardAgentDefense Category=Guardrail-based, Base Agent=Claude-3.52026.02 | 90 | 100 | 82.3 | 79.3 | 100 | |
| GuardAgentDefense Category=Guardrail-based, Base Agent=Qwen-max2026.02 | 83.8 | 91.6 | 72.8 | 79.6 | 100 | |
| Qwen-maxDefense Category=Model-based2026.02 | 82.4 | 94.7 | 40.8 | 70.6 | 100 | |
| gpt-oss-safeguard-20bDefense Category=Guardrail-based2026.02 | 80.6 | 83.2 | 83.1 | 84.3 | 98.3 | |
| Claude-3.5Defense Category=Model-based2026.02 | 78.6 | 86.9 | 100 | 85.1 | 100 | |
| LLaMA-Guard 3Defense Category=Guardrail-based2026.02 | 48.7 | — | 0 | — | — |