Safety Controllability on GenHarm
100Violence Refusal RateBase
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BaseAllowed Domains=None, Base Model=LLAMA2-7B-CHAT2026.05 | 100 | 100 | 100 | 100 | 100 | |
| BaseAllowed Domains=—, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 100 | 100 | 98.9 | 99.5 | 100 | |
| BaseAllowed Domains=None, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 100 | 100 | 98.9 | 99.5 | 100 | |
| BaseAllowed Domains=—2026.05 | 100 | 99.1 | 90.8 | 95.8 | 96.3 | |
| AutoDANAllowed Domains=Hate | Disinfo, Base Model=LLAMA2-7B-CHAT2026.05 | 98.8 | 98.1 | 78.5 | 97.8 | 98.8 | |
| AutoDANAllowed Domains=Disinfo | Sexual, Base Model=LLAMA2-7B-CHAT2026.05 | 98.8 | 98.1 | 78.5 | 97.8 | 98.8 | |
| PALETTEAllowed Domains=Hate | Disinfo2026.05 | 97.6 | 12.2 | 17.3 | 93.5 | 94.7 | |
| PALETTEAllowed Domains=Hate | Disinfo, Base Model=LLAMA2-7B-CHAT2026.05 | 96.4 | 6.6 | 12.2 | 94.7 | 90.7 | |
| AutoDANAllowed Domains=Violence | Hate, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 96.4 | 99.1 | 88.8 | 93.7 | 98.2 | |
| AutoDANAllowed Domains=Illegal | Violence, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 96.4 | 99.1 | 88.8 | 93.7 | 98.2 | |
| AutoDANAllowed Domains=Sexual | Illegal, Base Model=LLAMA2-7B-CHAT2026.05 | 95.2 | 81.1 | 97.9 | 85.5 | 96.3 | |
| PALETTEAllowed Domain=Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 95.2 | 15.1 | 76.5 | 90.5 | 90.7 | |
| PALETTEAllowed Domains=Hate | Disinfo, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 95.2 | 18.7 | 15.3 | 92.6 | 88.9 | |
| AutoDANAllowed Domains=Violence | Hate, Base Model=LLAMA2-7B-CHAT2026.05 | 93.9 | 97.1 | 63.2 | 97.8 | 79.6 | |
| AutoDANAllowed Domains=Illegal | Violence, Base Model=LLAMA2-7B-CHAT2026.05 | 93.9 | 97.1 | 63.2 | 97.8 | 79.6 | |
| BaseAllowed Domain=None, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 92.8 | 98.1 | 87.8 | 95.8 | 90.7 | |
| PALETTEAllowed Domain=Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 92.8 | 84.9 | 74.5 | 14.7 | 77.8 | |
| PALETTEAllowed Domain=Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 92.7 | 89.6 | 18.4 | 91.6 | 85.2 | |
| AutoDANAllowed Domains=Hate | Disinfo, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 92.7 | 97.2 | 61.2 | 98.9 | 83.3 | |
| AutoDANAllowed Domains=Disinfo | Sexual, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 92.7 | 97.2 | 61.2 | 98.9 | 83.3 | |
| AutoDANAllowed Domains=Sexual | Illegal, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 92.7 | 97.2 | 61.2 | 98.9 | 83.3 | |
| PALETTEAllowed Domains=Disinfo | Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 90.4 | 84.9 | 11.2 | 15.8 | 68.5 | |
| PALETTEAllowed Domains=Disinfo | Sexual2026.05 | 86.4 | 83.4 | 19.4 | 17.9 | 92.6 | |
| PALETTEAllowed Domains=Disinfo | Sexual, Base Model=LLAMA2-7B-CHAT2026.05 | 84.3 | 84.9 | 10.2 | 6.3 | 77.8 | |
| PALETTEAllowed Domains=Hate | Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 84.3 | 11.3 | 11.2 | 75.8 | 85.2 | |
| PALETTEAllowed Domains=Sexual | Illegal2026.05 | 84 | 95.4 | 89.2 | 15.9 | 14.8 | |
| CASTAllowed Domain=Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 83.1 | 27.4 | 74.5 | 78.9 | 87 | |
| CASTAllowed Domain=Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 83.1 | 60.4 | 55.1 | 38.9 | 88.9 | |
| PALETTEAllowed Domains=Sexual | Illegal, Base Model=LLAMA2-7B-CHAT2026.05 | 80.8 | 98.1 | 79.6 | 11.6 | 5.6 | |
| PALETTEAllowed Domains=Sexual | Illegal, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 79.5 | 87.7 | 76.5 | 23.1 | 12.9 | |
| PALETTEAllowed Domain=Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 79.3 | 96.2 | 84.7 | 81.1 | 17.7 | |
| PALETTEAllowed Domains=Disinfo | Sexual, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 73.5 | 68.9 | 3.1 | 5.3 | 70.4 | |
| PALETTEAllowed Domains=Sexual | Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 72.3 | 91.5 | 72.4 | 20 | 12.9 | |
| CASTAllowed Domains=Disinfo | Sexual2026.05 | 71.1 | 45.4 | 13.7 | 34.7 | 61.6 | |
| CASTAllowed Domains=Disinfo | Sexual, Base Model=LLAMA2-7B-CHAT2026.05 | 69.8 | 73 | 16.3 | 57.9 | 79.6 | |
| CASTAllowed Domains=Disinfo | Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 69.8 | 28.3 | 17.3 | 38.9 | 51.8 | |
| CASTAllowed Domains=Hate | Disinfo2026.05 | 69.3 | 38.8 | 11.3 | 58.3 | 78.4 | |
| CASTAllowed Domain=Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 65.1 | 56.6 | 14.3 | 66.3 | 79.6 | |
| CASTAllowed Domains=Hate | Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 63.9 | 48.1 | 14.3 | 58.9 | 70.4 | |
| CASTAllowed Domains=Sexual | Illegal2026.05 | 62.7 | 68.4 | 61.3 | 32.4 | 41.6 | |
| CASTAllowed Domains=Illegal | Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 61.5 | 72.6 | 87.8 | 71.6 | 40.7 | |
| CASTAllowed Domains=Disinfo | Sexual, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 60.2 | 43.4 | 5.1 | 41.1 | 88.9 | |
| CASTAllowed Domains=Sexual | Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 59 | 65.1 | 59.2 | 27.3 | 35.2 | |
| CASTAllowed Domains=Hate | Disinfo, Base Model=LLAMA2-7B-CHAT2026.05 | 57.8 | 52 | 6.1 | 43.1 | 64.8 | |
| CASTAllowed Domain=Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 51.8 | 50.9 | 78.6 | 57.9 | 29.6 | |
| CASTAllowed Domains=Hate | Disinfo, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 48.2 | 29.2 | 4.1 | 47.4 | 70.4 | |
| CASTAllowed Domains=Illegal | Violence, Base Model=LLAMA2-7B-CHAT2026.05 | 46.9 | 51 | 93.9 | 58.9 | 37 | |
| CASTAllowed Domain=Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 33.7 | 32.1 | 12.2 | 37.9 | 27.8 | |
| CASTAllowed Domains=Illegal | Violence2026.05 | 33.7 | 74.5 | 86.3 | 73.1 | 28.9 | |
| CASTAllowed Domains=Violence | Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 32.6 | 43.4 | 40.8 | 55.8 | 20.4 | |
| CASTAllowed Domains=Violence | Hate2026.05 | 28.3 | 35.2 | 42.5 | 51.9 | 30.5 | |
| PALETTEAllowed Domains=Illegal | Violence, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 21.7 | 90.6 | 82.7 | 76.8 | 5.6 | |
| PALETTEAllowed Domains=Illegal | Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 21.7 | 90.6 | 82.7 | 76.8 | 5.6 | |
| CASTAllowed Domains=Sexual | Illegal, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 19.3 | 23.6 | 72.4 | 25.3 | 12.9 | |
| PALETTEAllowed Domains=Violence | Hate2026.05 | 19.3 | 15.1 | 88.7 | 91.7 | 77.8 | |
| CASTAllowed Domains=Violence | Hate, Base Model=LLAMA2-7B-CHAT2026.05 | 18.1 | 11 | 58.2 | 23.1 | 9.2 | |
| CASTAllowed Domains=Illegal | Violence, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 16.8 | 69.8 | 84.6 | 42.1 | 12.9 | |
| PALETTEAllowed Domain=Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 14.6 | 92.5 | 85.7 | 82.1 | 70.4 | |
| PALETTEAllowed Domains=Illegal | Violence, Base Model=LLAMA2-7B-CHAT2026.05 | 13.2 | 91.5 | 75.5 | 71.6 | 9.3 | |
| PALETTEAllowed Domains=Violence | Hate, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 13.2 | 10.4 | 82.7 | 86.3 | 79.6 | |
| PALETTEAllowed Domains=Violence | Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 13.2 | 10.4 | 82.7 | 86.3 | 79.6 | |
| PALETTEAllowed Domains=Illegal | Violence2026.05 | 12 | 97.4 | 90.1 | 91.7 | 20.3 | |
| PALETTEAllowed Domains=Violence | Hate, Base Model=LLAMA2-7B-CHAT2026.05 | 9.6 | 3.8 | 82.7 | 85.3 | 64.8 | |
| CASTAllowed Domains=Violence | Hate, Backbone=LLAMA3-8B-INSTRUCT2026.05 | 9.6 | 11.3 | 44.9 | 17.9 | 9.2 | |
| CASTAllowed Domains=Sexual | Illegal, Base Model=LLAMA2-7B-CHAT2026.05 | 4.8 | 5.3 | 20.4 | 7.3 | 3.7 | |
| AutoDANAllowed Domains=Violence | Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 3.6 | 7.5 | 6.1 | 14.3 | 16.7 | |
| AutoDANAllowed Domains=Hate | Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 3.6 | 7.5 | 6.1 | 14.3 | 16.7 | |
| AutoDANAllowed Domains=Sexual | Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 3.6 | 7.5 | 6.1 | 14.3 | 16.7 | |
| AutoDANAllowed Domain=Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 2.4 | 0.9 | 1 | 4.2 | 5.6 | |
| AutoDANAllowed Domain=Illegal, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 2.4 | 0.9 | 1 | 4.2 | 5.6 | |
| AutoDANAllowed Domains=Violence | Hate2026.05 | 2.4 | 0.9 | 1 | 4.2 | 5.6 | |
| AutoDANAllowed Domains=Sexual | Illegal2026.05 | 2.4 | 0.9 | 1 | 4.2 | 5.6 | |
| AutoDANAllowed Domain=Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domain=Hate, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domain=Disinfo, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domains=Disinfo | Sexual, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domains=Illegal | Violence, Backbone=QWEN2.5-7B-INSTRUCT2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domains=Hate | Disinfo2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domains=Disinfo | Sexual2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 | |
| AutoDANAllowed Domains=Illegal | Violence2026.05 | 1.2 | 4.7 | 1 | 29.5 | 1.8 |