OS Assistance on Safe-OS system-sabotage and environment-dependent
100TSRbaseline
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| baselineAccess Control=LLM-AC, Core LLM=Gemini 2.5 Flash2026.06 | 100 | 10 | |
| FragFuseAccess Control=LLM-AC, Core LLM=Gemini 2.5 Flash2026.06 | 84.4 | 64 | |
| direct queryingAccess Control=None, Core LLM=GPT-5.12026.06 | 84 | — | |
| direct queryingAccess Control=None, Core LLM=Gemini 2.5 Flash2026.06 | 82 | — | |
| direct queryingAccess Control=None, Core LLM=Average2026.06 | 82 | — | |
| FragFuseAccess Control=LLM-AC, Core LLM=Average2026.06 | 81.4 | 80.7 | |
| FragFuseAccess Control=AGrail, Core LLM=GPT-5.12026.06 | 81.3 | 96 | |
| FragFuseAccess Control=AGrail, Core LLM=Gemini 2.5 Flash2026.06 | 80.9 | 94 | |
| FragFuseAccess Control=LLM-AC, Core LLM=GPT-4o2026.06 | 80.5 | 82 | |
| direct queryingAccess Control=None, Core LLM=GPT-4o2026.06 | 80 | — | |
| FragFuseAccess Control=LLM-AC, Core LLM=GPT-5.12026.06 | 79.2 | 96 | |
| FragFuseAccess Control=AGrail, Core LLM=Average2026.06 | 77.6 | 92.7 | |
| FragFuseAccess Control=AGrail, Core LLM=GPT-4o2026.06 | 70.5 | 88 | |
| baselineAccess Control=AGrail, Core LLM=GPT-4o2026.06 | 40 | 10 | |
| baselineAccess Control=LLM-AC, Core LLM=Average2026.06 | 33.3 | 3.3 | |
| baselineAccess Control=AGrail, Core LLM=Average2026.06 | 21.4 | 19.3 | |
| baselineAccess Control=AGrail, Core LLM=GPT-5.12026.06 | 14.3 | 28 | |
| baselineAccess Control=AGrail, Core LLM=Gemini 2.5 Flash2026.06 | 10 | 20 | |
| baselineAccess Control=LLM-AC, Core LLM=GPT-4o2026.06 | 0 | 0 | |
| baselineAccess Control=LLM-AC, Core LLM=GPT-5.12026.06 | 0 | 0 |