Secure tool generalization on ToolEmu and SafeToolBench Balanced OOD transfer
56.9GUARDEDJOINT ScoreVISTA-Guard
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| VISTA-Guardevaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 56.9 | 2.4 | 25.5 | |
| TF-IDF + LogRegevaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 50.3 | 1.6 | 31 | |
| TF-IDF + SVMevaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 43.6 | 0.3 | 37.2 | |
| ToolShieldevaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 43.3 | 8.3 | 26.7 | |
| GB (10-feat)evaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 7.4 | 26.8 | 26 | |
| TF-IDF + MLPevaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 0 | 32 | 31.6 | |
| BERT-baseevaluation_protocol=Balanced OOD transfer, rho=1.52026.05 | 0 | 73.3 | 2.7 |