Harmfulness Detection on BeaverTails bottom 30% uncertainty slice (test)
85.1AUROCGeometry-Lite
Evaluation results
| Method | Links | |
|---|---|---|
| Geometry-LiteBackbone=Qwen3-14B2026.05 | 85.1 | |
| MultiLayer-LinearBackbone=Qwen3-1.7B2026.05 | 83.8 | |
| Geometry-LiteBackbone=Qwen3-1.7B2026.05 | 83.8 | |
| Geometry-LiteBackbone=Llama-70B2026.05 | 83.7 | |
| MultiLayer-LinearBackbone=Llama-70B2026.05 | 83.6 | |
| Geometry-LiteBackbone=Llama-8B2026.05 | 83.4 | |
| MultiLayer-LinearBackbone=Llama-1B2026.05 | 83 | |
| MultiLayer-LinearBackbone=Llama-8B2026.05 | 82.9 | |
| MultiLayer-LinearBackbone=Qwen3-14B2026.05 | 82.7 | |
| MultiLayer-LinearBackbone=Avg2026.05 | 82.7 | |
| Geometry-LiteBackbone=Llama-1B2026.05 | 82.6 | |
| Geometry-LiteBackbone=Avg2026.05 | 82.6 | |
| TaT-Disp-LSTMBackbone=Qwen3-14B2026.05 | 82.5 | |
| MultiLayer-LinearBackbone=Qwen3-8B2026.05 | 82.4 | |
| TaT-Disp-LSTMBackbone=Qwen3-1.7B2026.05 | 82.3 | |
| MultiLayer-LinearBackbone=Gemma-9B2026.05 | 82.3 | |
| Geometry-LiteBackbone=Qwen3-8B2026.05 | 82.2 | |
| MultiLayer-LinearBackbone=Gemma-2B2026.05 | 82 | |
| MultiLayer-LinearBackbone=Gemma-27B2026.05 | 81.9 | |
| Geometry-LiteBackbone=Gemma-2B2026.05 | 81 | |
| Geometry-LiteBackbone=Gemma-9B2026.05 | 81 | |
| Geometry-LiteBackbone=Gemma-27B2026.05 | 80.7 | |
| Best single-layer probeBackbone=Llama-70B2026.05 | 80.6 | |
| Final-layer probeBackbone=Llama-70B2026.05 | 80 | |
| Refusal-style probeBackbone=Llama-70B2026.05 | 79.9 | |
| TaT-Disp-LSTMBackbone=Llama-1B2026.05 | 79.8 | |
| TaT-Disp-LSTMBackbone=Qwen3-8B2026.05 | 79.5 | |
| TaT-Disp-LSTMBackbone=Avg2026.05 | 79.4 | |
| TaT-Disp-LSTMBackbone=Gemma-2B2026.05 | 79.1 | |
| Final-layer probeBackbone=Llama-8B2026.05 | 78.9 | |
| TaT-Disp-LSTMBackbone=Llama-8B2026.05 | 78.3 | |
| TaT-Disp-LSTMBackbone=Gemma-9B2026.05 | 78.1 | |
| TaT-Disp-LSTMBackbone=Llama-70B2026.05 | 77.4 | |
| TaT-Disp-LSTMBackbone=Gemma-27B2026.05 | 77.4 | |
| Best single-layer probeBackbone=Gemma-9B2026.05 | 77.3 | |
| Final-layer probeBackbone=Qwen3-14B2026.05 | 77.1 | |
| Best single-layer probeBackbone=Llama-8B2026.05 | 76.9 | |
| MultiLayer-DIMBackbone=Gemma-2B2026.05 | 76.6 | |
| Best single-layer probeBackbone=Gemma-27B2026.05 | 76.3 | |
| Refusal-style probeBackbone=Llama-8B2026.05 | 76.3 | |
| Final-layer probeBackbone=Llama-1B2026.05 | 76.1 | |
| Final-layer probeBackbone=Avg2026.05 | 76.1 | |
| Best single-layer probeBackbone=Avg2026.05 | 75.8 | |
| Best single-layer probeBackbone=Gemma-2B2026.05 | 75.7 | |
| Refusal-style probeBackbone=Qwen3-1.7B2026.05 | 75.7 | |
| Final-layer probeBackbone=Gemma-27B2026.05 | 75.5 | |
| MultiLayer-DIMBackbone=Gemma-9B2026.05 | 75.2 | |
| Final-layer probeBackbone=Gemma-9B2026.05 | 75 | |
| Final-layer probeBackbone=Qwen3-1.7B2026.05 | 75 | |
| Final-layer probeBackbone=Gemma-2B2026.05 | 74.9 | |
| Best single-layer probeBackbone=Qwen3-8B2026.05 | 74.9 | |
| Best single-layer probeBackbone=Qwen3-14B2026.05 | 74.8 | |
| Refusal-style probeBackbone=Llama-1B2026.05 | 74.5 | |
| Refusal-style probeBackbone=Avg2026.05 | 74.1 | |
| Best single-layer probeBackbone=Llama-1B2026.05 | 73.5 | |
| Refusal-style probeBackbone=Qwen3-8B2026.05 | 73.5 | |
| Refusal-style probeBackbone=Gemma-27B2026.05 | 73.1 | |
| Refusal-style probeBackbone=Gemma-9B2026.05 | 72.9 | |
| Final-layer probeBackbone=Qwen3-8B2026.05 | 72.8 | |
| MultiLayer-DIMBackbone=Gemma-27B2026.05 | 72.5 | |
| MultiLayer-DIMBackbone=Qwen3-14B2026.05 | 72.3 | |
| Best single-layer probeBackbone=Qwen3-1.7B2026.05 | 72.2 | |
| MultiLayer-DIMBackbone=Llama-70B2026.05 | 72 | |
| MultiLayer-DIMBackbone=Avg2026.05 | 72 | |
| Refusal-style probeBackbone=Qwen3-14B2026.05 | 70.9 | |
| MultiLayer-DIMBackbone=Llama-8B2026.05 | 70.3 | |
| MultiLayer-DIMBackbone=Llama-1B2026.05 | 69.9 | |
| MultiLayer-DIMBackbone=Qwen3-1.7B2026.05 | 69.8 | |
| MultiLayer-DIMBackbone=Qwen3-8B2026.05 | 69.7 | |
| Refusal-style probeBackbone=Gemma-2B2026.05 | 69.6 |