Coherence evaluation on HarmBench (test)
0.22N-gram Repetition RateDirAbl
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DirAblBackbone=gemma-2-9b, Steering Angles=Averaged2026.01 | 0.22 | 100 | 53.25 | |
| DirAblBackbone=gemma-2-2b, Steering Angles=Averaged2026.01 | 1.38 | 99.99 | 47.21 | |
| DirAblBackbone=Qwen2.5-7B, Steering Angles=Averaged2026.01 | 1.58 | 99.95 | 46.95 | |
| DirAblBackbone=Llama-3.1-8B, Steering Angles=Averaged2026.01 | 1.82 | 99.99 | 69.73 | |
| DirAblBackbone=Qwen2.5-3B, Steering Angles=Averaged2026.01 | 2.19 | 99.96 | 46.21 | |
| DirAblBackbone=Llama-3.2-3B, Steering Angles=Averaged2026.01 | 3.21 | 100 | 55.88 | |
| DirAblBackbone=Llama-3.2-1B, Steering Angles=Averaged2026.01 | 4.17 | 99.98 | 51.31 | |
| DirAblBackbone=Qwen2.5-1.5B, Steering Angles=Averaged2026.01 | 5.07 | 99.99 | 52.78 | |
| ActAddBackbone=Llama-3.1-8B, Steering Angles=Averaged2026.01 | 7.25 | 100 | 42.74 | |
| ActAddBackbone=Llama-3.2-3B, Steering Angles=Averaged2026.01 | 7.59 | 100 | 41.15 | |
| ActAddBackbone=Qwen2.5-7B, Steering Angles=Averaged2026.01 | 13.77 | 99.91 | 39.48 | |
| ActAddBackbone=Qwen2.5-1.5B, Steering Angles=Averaged2026.01 | 18.49 | 30.93 | 21.92 | |
| ActAddBackbone=Llama-3.2-1B, Steering Angles=Averaged2026.01 | 19.83 | 100 | 39.67 | |
| ActAddBackbone=Qwen2.5-3B, Steering Angles=Averaged2026.01 | 46.23 | 99.98 | 23.3 | |
| ActAddBackbone=gemma-2-9b, Steering Angles=Averaged2026.01 | 97.07 | 100 | 7.53 | |
| ActAddBackbone=gemma-2-2b, Steering Angles=Averaged2026.01 | 98.04 | 100 | 3.2 |