Hijacking detection on Tool-input hijacking detection dataset 20% (test)
100AUC-ROCProbing
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ProbingModel=Qwen-32B, Layer=532026.04 | 100 | 100 | 100 | |
| Cosine SimilarityModel=Gemma3-12B, Layer=52026.04 | 100 | 100 | 100 | |
| ProbingModel=Mistral, Layer=22026.04 | 100 | 100 | 100 | |
| ProbingModel=Qwen-14B, Layer=312026.04 | 99.97 | 99.97 | 100 | |
| Cosine SimilarityModel=Qwen3-4B, Layer=42026.04 | 99.68 | 99.7 | 96 | |
| ProbingModel=Qwen3-8B, Layer=222026.04 | 99.63 | 99.61 | 97.39 | |
| ProbingModel=Qwen3-14B, Layer=182026.04 | 99.31 | 99.39 | 97.12 | |
| ProbingModel=LLaMA3-8B, Layer=52026.04 | 99.2 | 99.51 | 98.68 | |
| ProbingModel=Qwen3-4B, Layer=42026.04 | 99.12 | 99.15 | 93.39 | |
| ProbingModel=GLM4-9B, Layer=232026.04 | 98.3 | 98.32 | 87.03 | |
| Cosine SimilarityModel=Qwen3-8B, Layer=222026.04 | 88.66 | 90.25 | 63.77 | |
| ProbingModel=Gemma3-12B, Layer=52026.04 | 87.76 | 93.41 | 85.71 | |
| Cosine SimilarityModel=Qwen-14B, Layer=312026.04 | 87.05 | 89.97 | 66.28 | |
| Cosine SimilarityModel=GLM4-9B, Layer=232026.04 | 86.01 | 83.96 | 26.13 | |
| Cosine SimilarityModel=Mistral, Layer=22026.04 | 85.6 | 87.91 | 57.89 | |
| Cosine SimilarityModel=Qwen-32B, Layer=532026.04 | 83.79 | 88.36 | 70.11 | |
| Cosine SimilarityModel=LLaMA3-8B, Layer=52026.04 | 79.21 | 82.65 | 36.96 | |
| Cosine SimilarityModel=Qwen3-14B, Layer=182026.04 | 75.94 | 78.69 | 51.19 |