Safe-or-harmful binary classification on BeaverTails
84.6AccuracyTELLME
Evaluation Results
| Method | Links | |
|---|---|---|
| TELLMEBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 84.6 | |
| TELLMEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 84.6 | |
| TELLMEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 84.5 | |
| SAE-KLBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 84.4 | |
| SAE-Multi-TopKBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 84.4 | |
| SAEBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 84.2 | |
| TELLMEBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 83.8 | |
| Skip TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 83.1 | |
| OriginBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 82.9 | |
| SAEBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 82.7 | |
| TELLMEBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 82.5 | |
| SAEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 82.5 | |
| SAE-KLBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 82.5 | |
| TELLMEBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 82.5 | |
| Skip TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 82.3 | |
| SAE-Multi-TopKBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 82.3 | |
| SAE-KLBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 82.1 | |
| TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 82.1 | |
| TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 82 | |
| Skip TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 81.6 | |
| SAE-Multi-TopKBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 81.5 | |
| OriginBackbone=Mistral-7B-v0.3, Evaluation Protocol=Linear Probe2025.02 | 81.3 | |
| TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Linear Probe2025.02 | 81.2 | |
| TELLMEBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 76.9 | |
| Skip TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 75.6 | |
| TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 75.5 | |
| OriginBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 75.4 | |
| OriginBackbone=Qwen2.5-7B, Evaluation Protocol=Linear Probe2025.02 | 75.3 | |
| SAEBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 75.2 | |
| SAE-Multi-TopKBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 75.2 | |
| TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 75.2 | |
| Skip TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 75.1 | |
| SAE-KLBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 74.8 | |
| SAEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 74.7 | |
| SAE-Multi-TopKBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 74.7 | |
| SAE-KLBackbone=Mistral-7B-v0.3, Evaluation Protocol=Self-Sim2025.02 | 74.6 | |
| SAE-KLBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 73.8 | |
| SAEBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 73.7 | |
| OriginBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 73.4 | |
| SAE-Multi-TopKBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 73.3 | |
| Skip TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 73.1 | |
| OriginBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 73.1 | |
| SAE-KLBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 73 | |
| TranscoderBackbone=Llama-3.1-8B, Evaluation Protocol=Self-Sim2025.02 | 72.9 | |
| SAEBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 72.8 | |
| TELLMEBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 72.7 | |
| SAE-Multi-TopKBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 72.1 | |
| Skip TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 72 | |
| SAEBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 71.9 | |
| SAE-KLBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 71.8 | |
| SAE-Multi-TopKBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 71.8 | |
| TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 71.8 | |
| OriginBackbone=Llama-3.1-8B, Evaluation Protocol=Judge2025.02 | 71.6 | |
| OriginBackbone=Qwen2.5-7B, Evaluation Protocol=Judge2025.02 | 71.6 | |
| TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 70.4 | |
| Skip TranscoderBackbone=Qwen2.5-7B, Evaluation Protocol=Self-Sim2025.02 | 70.4 | |
| TELLMEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 65 | |
| OriginBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 63.7 | |
| SAE-Multi-TopKBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 55.6 | |
| SAE-KLBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 55.2 | |
| SAEBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 53.6 | |
| Skip TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 53.4 | |
| TranscoderBackbone=Mistral-7B-v0.3, Evaluation Protocol=Judge2025.02 | 52.8 |