Tool-use on AgentHarm (public test)
50TSRbaseline
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| baselineAccess Control=LLM-AC, Core LLM=GPT-4o2026.06 | 50 | 3.4 | |
| direct queryingAccess Control=None, Core LLM=GPT-4o2026.06 | 38.6 | — | |
| baselineAccess Control=LLM-AC, Core LLM=GPT-5.12026.06 | 32.4 | 21 | |
| baselineAccess Control=ShieldAgent, Core LLM=Gemini 2.5 Flash2026.06 | 28.9 | 21.6 | |
| baselineAccess Control=LLM-AC, Core LLM=Average2026.06 | 27.5 | 8.1 | |
| direct queryingAccess Control=None, Core LLM=Average2026.06 | 25 | — | |
| FragFuseAccess Control=LLM-AC, Core LLM=GPT-4o2026.06 | 25 | 45.5 | |
| direct queryingAccess Control=None, Core LLM=Gemini 2.5 Flash2026.06 | 22.7 | — | |
| FragFuseAccess Control=ShieldAgent, Core LLM=Gemini 2.5 Flash2026.06 | 19.3 | 85.2 | |
| FragFuseAccess Control=ShieldAgent, Core LLM=GPT-4o2026.06 | 17.2 | 82.4 | |
| direct queryingAccess Control=None, Core LLM=GPT-5.12026.06 | 13.6 | — | |
| FragFuseAccess Control=ShieldAgent, Core LLM=Average2026.06 | 13.1 | 88.3 | |
| baselineAccess Control=ShieldAgent, Core LLM=Average2026.06 | 13 | 18.7 | |
| FragFuseAccess Control=LLM-AC, Core LLM=Average2026.06 | 10.9 | 80.5 | |
| baselineAccess Control=ShieldAgent, Core LLM=GPT-5.12026.06 | 10.2 | 33.5 | |
| FragFuseAccess Control=LLM-AC, Core LLM=GPT-5.12026.06 | 7.6 | 97.2 | |
| FragFuseAccess Control=ShieldAgent, Core LLM=GPT-5.12026.06 | 2.9 | 97.2 | |
| baselineAccess Control=LLM-AC, Core LLM=Gemini 2.5 Flash2026.06 | 0 | 0 | |
| FragFuseAccess Control=LLM-AC, Core LLM=Gemini 2.5 Flash2026.06 | 0 | 98.9 | |
| baselineAccess Control=ShieldAgent, Core LLM=GPT-4o2026.06 | 0 | 1.1 |