Multi-risk safety monitoring on BeaverTails
80.1Accuracy (%)TELLME
Evaluation Results
| Method | Links | |
|---|---|---|
| TELLMEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 80.1 | |
| TELLMEBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 79.1 | |
| TELLMEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 79 | |
| TELLMEBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 78.8 | |
| OriginBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 77.8 | |
| SAE-KLBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 77.6 | |
| OriginBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 77.4 | |
| SAEBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 77.3 | |
| SAE-Multi-TopKBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 77 | |
| SAEBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 76.6 | |
| SAE-Multi-TopKBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 76.2 | |
| SAE-KLBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 76.2 | |
| SAEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 76.1 | |
| SAE-KLBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 75.8 | |
| TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 75.8 | |
| SAE-Multi-TopKBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 75.8 | |
| OriginBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 75 | |
| Skip TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 74.6 | |
| TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 73 | |
| TELLMEBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 73 | |
| Skip TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 71.8 | |
| TELLMEBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 70.8 | |
| Skip TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 69.3 | |
| TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 69.3 | |
| TELLMEBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 61.5 | |
| Skip TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 59.8 | |
| TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 59.7 | |
| OriginBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 59.2 | |
| OriginBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 59.1 | |
| SAEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 59 | |
| SAE-Multi-TopKBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 59 | |
| SAE-KLBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 58.8 | |
| TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 57.7 | |
| Skip TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 57.6 | |
| Skip TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 57.4 | |
| TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 57.1 | |
| SAEBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 57 | |
| SAE-KLBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 56.8 | |
| SAE-KLBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 56.4 | |
| SAEBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 56.3 | |
| OriginBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 56.1 | |
| SAE-Multi-TopKBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 56.1 | |
| SAE-Multi-TopKBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 55.6 | |
| TELLMEBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 54.4 | |
| SAE-KLBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 54.3 | |
| OriginBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 54.2 | |
| SAEBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 54.2 | |
| SAE-KLBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 54.1 | |
| SAEBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 53.8 | |
| SAE-Multi-TopKBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 53.5 | |
| SAE-Multi-TopKBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 53.2 | |
| TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 52.4 | |
| Skip TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 52.4 | |
| OriginBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 52.3 | |
| Skip TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 52.3 | |
| TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 51.5 | |
| OriginBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 36.7 | |
| SAE-Multi-TopKBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 23 | |
| TELLMEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 22.8 | |
| SAE-KLBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 20.2 | |
| SAEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 19.1 | |
| Skip TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 16.9 | |
| TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 16.7 |