Safety Evaluation on BeaverTails (test) using Harmful Score
7.9Harmful ScoreSafeInstruct
Evaluation Results
| Method | Links | |
|---|---|---|
| SafeInstructHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 7.9 | |
| Buffer-and-ReinforceHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 8.1 | |
| Buffer-and-Reinforceuser data size (n)=15002026.05 | 8.2 | |
| Buffer-and-ReinforceHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 8.2 | |
| Buffer-and-Reinforceuser data size (n)=10002026.05 | 8.4 | |
| Buffer-and-Reinforceuser data size (n)=5002026.05 | 8.5 | |
| Buffer-and-Reinforceuser data size (n)=Average2026.05 | 8.6 | |
| Buffer-and-Reinforceuser data size (n)=20002026.05 | 8.7 | |
| Buffer-and-ReinforceHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 8.7 | |
| Buffer-and-ReinforceHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 8.7 | |
| Buffer-and-ReinforceHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 8.8 | |
| Buffer-and-Reinforceuser data size (n)=25002026.05 | 9.1 | |
| LisaHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 14.7 | |
| LDIFSHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 16.4 | |
| LDIFSuser data size (n)=20002026.05 | 16.5 | |
| LDIFSHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 16.5 | |
| LDIFSHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 16.6 | |
| LDIFSHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 16.9 | |
| LDIFSuser data size (n)=10002026.05 | 17.2 | |
| LDIFSHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 17.6 | |
| LDIFSuser data size (n)=Average2026.05 | 18 | |
| LDIFSuser data size (n)=15002026.05 | 18.4 | |
| LDIFSuser data size (n)=25002026.05 | 18.7 | |
| SafeInstructuser data size (n)=15002026.05 | 19 | |
| SafeInstructuser data size (n)=25002026.05 | 19 | |
| LDIFSuser data size (n)=5002026.05 | 19.1 | |
| SafeInstructuser data size (n)=20002026.05 | 19.4 | |
| SafeInstructHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 19.6 | |
| PanaceaHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 19.9 | |
| SafeLoRAHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 20 | |
| Security Vectoruser data size (n)=5002026.05 | 20.5 | |
| Security Vectoruser data size (n)=20002026.05 | 21 | |
| AsFTHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 21.1 | |
| SafeInstructuser data size (n)=10002026.05 | 21.5 | |
| SafeInstructuser data size (n)=Average2026.05 | 21.6 | |
| Security Vectoruser data size (n)=10002026.05 | 21.7 | |
| SafeLoRAuser data size (n)=5002026.05 | 21.7 | |
| Security Vectoruser data size (n)=Average2026.05 | 22 | |
| Security VectorHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 22.1 | |
| AntidoteHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 22.2 | |
| Security VectorHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 22.8 | |
| Lisauser data size (n)=25002026.05 | 23.1 | |
| Security Vectoruser data size (n)=25002026.05 | 23.2 | |
| Security Vectoruser data size (n)=15002026.05 | 23.6 | |
| AsFTuser data size (n)=5002026.05 | 23.7 | |
| Security VectorHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 23.9 | |
| Security VectorHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 24.3 | |
| Lisauser data size (n)=20002026.05 | 24.7 | |
| SafeLoRAuser data size (n)=20002026.05 | 24.8 | |
| Lisauser data size (n)=Average2026.05 | 25.2 | |
| SafeLoRAuser data size (n)=Average2026.05 | 25.2 | |
| Antidoteuser data size (n)=10002026.05 | 25.3 | |
| Security VectorHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 25.3 | |
| Lisauser data size (n)=5002026.05 | 25.5 | |
| SafeLoRAuser data size (n)=15002026.05 | 25.9 | |
| Lisauser data size (n)=15002026.05 | 26 | |
| AsFTuser data size (n)=10002026.05 | 26 | |
| SafeLoRAuser data size (n)=10002026.05 | 26 | |
| SafeLoRAHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 26.6 | |
| AsFTHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 26.7 | |
| Lisauser data size (n)=10002026.05 | 26.8 | |
| AntidoteHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 27.2 | |
| Antidoteuser data size (n)=5002026.05 | 27.4 | |
| SafeLoRAuser data size (n)=25002026.05 | 27.6 | |
| AsFTuser data size (n)=Average2026.05 | 28.3 | |
| AsFTuser data size (n)=15002026.05 | 28.8 | |
| SafeInstructuser data size (n)=5002026.05 | 29 | |
| AsFTHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 29.6 | |
| AsFTuser data size (n)=20002026.05 | 29.7 | |
| LisaHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 29.9 | |
| SafeLoRAHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 31.7 | |
| AsFTHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 32.2 | |
| SFTHarmful data ratio (p)=0.0, Backbone=LLaMA3-8B-Instruct2026.05 | 33 | |
| AsFTuser data size (n)=25002026.05 | 33.2 | |
| AsFTHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 34.1 | |
| Panaceauser data size (n)=5002026.05 | 35.7 | |
| PanaceaHarmful data ratio (p)=0.1, Backbone=LLaMA3-8B-Instruct2026.05 | 36.2 | |
| Panaceauser data size (n)=10002026.05 | 41 | |
| Antidoteuser data size (n)=Average2026.05 | 41.1 | |
| SafeLoRAHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 41.6 | |
| AntidoteHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 45.4 | |
| Antidoteuser data size (n)=15002026.05 | 45.6 | |
| PanaceaHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 47.4 | |
| AntidoteHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 49.8 | |
| LisaHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 49.9 | |
| Antidoteuser data size (n)=20002026.05 | 50.4 | |
| SafeInstructHarmful data ratio (p)=0.3, Backbone=LLaMA3-8B-Instruct2026.05 | 50.5 | |
| PanaceaHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 52.5 | |
| SafeLoRAHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 53.2 | |
| Antidoteuser data size (n)=25002026.05 | 56.7 | |
| AntidoteHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 58.1 | |
| SFTuser data size (n)=5002026.05 | 61.8 | |
| Panaceauser data size (n)=Average2026.05 | 62 | |
| Panaceauser data size (n)=25002026.05 | 62.9 | |
| LisaHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 64 | |
| PanaceaHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 64.2 | |
| SafeInstructHarmful data ratio (p)=0.5, Backbone=LLaMA3-8B-Instruct2026.05 | 66.3 | |
| LisaHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 73.5 | |
| SafeInstructHarmful data ratio (p)=1.0, Backbone=LLaMA3-8B-Instruct2026.05 | 74 | |
| SFTuser data size (n)=Average2026.05 | 74.7 |