Safety Control on WMDP and GenHarm (test)
2.2Refusal Rate (Chem)CAST
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CASTAllowed Domain=Cyber, Base Model=LLaMA3-8B-Instruct2026.05 | 2.2 | 12.3 | 4.1 | 71.8 | 0.667 | |
| PALETTEAllowed Domain=Chem, Base Model=LLaMA3-8B-Instruct2026.05 | 7.2 | 75.2 | 98.7 | 97.7 | 0.692 | |
| CASTAllowed Domain=Chem, Base Model=LLaMA3-8B-Instruct2026.05 | 22.5 | 46.1 | 49.6 | 79.1 | 0.672 | |
| CASTAllowed Domain=Bio, Base Model=LLaMA3-8B-Instruct2026.05 | 34.8 | 35.9 | 41 | 73.9 | 0.673 | |
| AutoDANAllowed Domain=Bio, Base Model=LLaMA3-8B-Instruct2026.05 | 84.9 | 82.2 | 91.4 | 92.4 | 0.421 | |
| PALETTEAllowed Domain=Bio, Base Model=LLaMA3-8B-Instruct2026.05 | 87.7 | 10.1 | 97.5 | 95.4 | 0.688 | |
| AutoDANAllowed Domain=Chem, Base Model=LLaMA3-8B-Instruct2026.05 | 90.6 | 83.1 | 93 | 94.5 | 0.441 | |
| AutoDANAllowed Domain=Cyber, Base Model=LLaMA3-8B-Instruct2026.05 | 90.6 | 83.1 | 93 | 94.5 | 0.441 | |
| PALETTEAllowed Domain=Cyber, Base Model=LLaMA3-8B-Instruct2026.05 | 93.5 | 85.4 | 2.9 | 85.2 | 0.686 | |
| BaseAllowed Domain=-, Base Model=LLaMA3-8B-Instruct2026.05 | 96.4 | 93.3 | 98 | 99.3 | 0.698 |