Language Understanding and Reasoning on MMLU, TruthfulQA, HellaSwag, and ARC-Easy (test)
0.082MMLU ScoreUnlearn-Smooth
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Unlearn-SmoothModel=OPTML-Group/NPO-SAM-WMDP, Attack=Adaptive attack2026.05 | 0.082 | 0.034 | 0.48 | 0.457 | 0.263 | |
| VAAModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.071 | 0.008 | 0.072 | 0.126 | 0.065 | |
| CTRAPModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.029 | 0.017 | 0.047 | 0.058 | 0.029 | |
| SDDModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.011 | 0.048 | 0.024 | 0.018 | 0.016 | |
| SDDModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.011 | 0.045 | 0.001 | 0.019 | 0.009 | |
| BoosterModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.01 | 0.011 | 0.024 | 0.037 | 0.015 | |
| BoosterModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.01 | 0.001 | 0.02 | 0.014 | 0.006 | |
| BoosterModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.008 | 0.006 | 0.013 | 0.035 | 0.009 | |
| VAAModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.007 | 0.101 | 0.004 | 0.038 | 0.018 | |
| VaccineModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.007 | 0.05 | 0.009 | 0.138 | 0.026 | |
| VaccineModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.007 | 0.092 | 0.004 | 0.034 | 0.017 | |
| CTRAPModel=Llama-3.1-8B, Attack=Adaptive attack2026.05 | 0.006 | 0.007 | 0.023 | 0.054 | 0.016 | |
| SDDModel=Qwen2-7B, Attack=Adaptive attack2026.05 | 0.004 | 0.021 | 0.012 | 0.02 | 0.002 | |
| VAAModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.003 | 0.069 | 0.003 | 0.054 | 0.002 | |
| CTRAPModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.001 | 0.004 | 0.036 | 0.061 | 0.023 | |
| VaccineModel=Llama-2-7B, Attack=Adaptive attack2026.05 | 0.001 | 0.1 | 0.025 | 0.054 | 0.018 |