Utility Retention on MMLU and GSM8K
47.3MMLU ScoreBase
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaseAllowed Domains=None, Base Model=LLAMA2-7B-CHAT2026.05 | 47.3 | 25.7 | 36.5 | |
| CASTAllowed Domains=Hate | Disinfo, Base Model=LLAMA2-7B-CHAT2026.05 | 47.2 | 20.3 | 33.7 | |
| PALETTEAllowed Domains=Hate | Disinfo, Base Model=LLAMA2-7B-CHAT2026.05 | 46.9 | 23.2 | 35.1 | |
| PALETTEAllowed Domains=Violence | Hate, Base Model=LLAMA2-7B-CHAT2026.05 | 46.8 | 25.7 | 36.3 | |
| PALETTEAllowed Domains=Disinfo | Sexual, Base Model=LLAMA2-7B-CHAT2026.05 | 46.3 | 24.3 | 35.3 | |
| PALETTEAllowed Domains=Illegal | Violence, Base Model=LLAMA2-7B-CHAT2026.05 | 45.8 | 21.8 | 33.8 | |
| PALETTEAllowed Domains=Sexual | Illegal, Base Model=LLAMA2-7B-CHAT2026.05 | 45.6 | 23.1 | 34.3 | |
| CASTAllowed Domains=Sexual | Illegal, Base Model=LLAMA2-7B-CHAT2026.05 | 45.4 | 19.4 | 32.4 | |
| CASTAllowed Domains=Violence | Hate, Base Model=LLAMA2-7B-CHAT2026.05 | 45.3 | 21.9 | 33.6 | |
| CASTAllowed Domains=Illegal | Violence, Base Model=LLAMA2-7B-CHAT2026.05 | 45.1 | 20.9 | 33 | |
| CASTAllowed Domains=Disinfo | Sexual, Base Model=LLAMA2-7B-CHAT2026.05 | 44.6 | 21.7 | 33.2 | |
| AutoDANAllowed Domains=Sexual | Illegal, Base Model=LLAMA2-7B-CHAT2026.05 | 26.4 | 7.2 | 16.8 | |
| AutoDANAllowed Domains=Hate | Disinfo, Base Model=LLAMA2-7B-CHAT2026.05 | 24.3 | 12.5 | 18.4 | |
| AutoDANAllowed Domains=Disinfo | Sexual, Base Model=LLAMA2-7B-CHAT2026.05 | 24.3 | 12.5 | 18.4 | |
| AutoDANAllowed Domains=Violence | Hate, Base Model=LLAMA2-7B-CHAT2026.05 | 23.2 | 6.4 | 14.8 | |
| AutoDANAllowed Domains=Illegal | Violence, Base Model=LLAMA2-7B-CHAT2026.05 | 23.2 | 6.4 | 14.8 |