Language Model Robustness under Adversarial Attack on MMLU Adversarially Perturbed
43.5AuA (GCG)Adap. RMU
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Adap. RMUUnlearning Framework=Representation Misdirection, Protocol=w/ RNA2025.01 | 43.5 | 62.4 | 30.8 | 75.2 | 39.2 | 72.7 | 50.8 | 79.2 | |
| NPO+MSEUnlearning Framework=Preference Optimization, Protocol=Original2025.01 | 40.7 | 57.6 | 29.1 | 66.7 | 40.3 | 66 | 46.6 | 71.2 | |
| BaseProtocol=Original2025.01 | 40.3 | — | 33.6 | — | 39.6 | — | 52.9 | — | |
| RMUUnlearning Framework=Representation Misdirection, Protocol=w/ RNA2025.01 | 40.3 | 60.9 | 30.5 | 79.1 | 38.9 | 76.4 | 50.1 | 84.2 | |
| RSVUnlearning Framework=Representation Misdirection, Protocol=Original2025.01 | 39.2 | 63 | 30.8 | 78.2 | 38.5 | 75.3 | 48.7 | 80.6 | |
| Adap. RMUUnlearning Framework=Representation Misdirection, Protocol=Original2025.01 | 38.5 | 63.5 | 30.5 | 80 | 38.9 | 76.2 | 49.8 | 83.3 | |
| RSVUnlearning Framework=Representation Misdirection, Protocol=w/ RNA2025.01 | 38.2 | 62.5 | 27 | 77.1 | 35 | 73.7 | 44.2 | 83.2 | |
| SimNPO+MSEUnlearning Framework=Preference Optimization, Protocol=w/ RNA2025.01 | 38.2 | 58.6 | 31.5 | 71.9 | 42.8 | 70.1 | 48.4 | 75.7 | |
| NPO+KLUnlearning Framework=Preference Optimization, Protocol=Original2025.01 | 35.4 | 51.2 | 26.3 | 61.3 | 34.7 | 61.3 | 43.1 | 67.7 | |
| SimNPO+MSEUnlearning Framework=Preference Optimization, Protocol=Original2025.01 | 35.4 | 52.3 | 29.8 | 66.7 | 36.8 | 66.8 | 44.5 | 72.3 | |
| NPO+MSEUnlearning Framework=Preference Optimization, Protocol=w/ RNA2025.01 | 34.3 | 51.6 | 24.2 | 66.2 | 34.3 | 64.4 | 46.3 | 70.9 | |
| RMUUnlearning Framework=Representation Misdirection, Protocol=Original2025.01 | 33.6 | 63 | 30.5 | 81.2 | 38.2 | 76.8 | 50.5 | 85.2 | |
| NPO+KLUnlearning Framework=Preference Optimization, Protocol=w/ RNA2025.01 | 31.2 | 55 | 25.2 | 64.1 | 31.2 | 62.3 | 39.2 | 67 | |
| DPO+KLUnlearning Framework=Preference Optimization, Protocol=Original2025.01 | 30.1 | 47.3 | 27.7 | 58.1 | 34 | 57.3 | 41.7 | 61.1 | |
| SimNPO+KLUnlearning Framework=Preference Optimization, Protocol=w/ RNA2025.01 | 30.1 | 55 | 27 | 61.7 | 35.4 | 60.8 | 44.2 | 67 | |
| DPO+KLUnlearning Framework=Preference Optimization, Protocol=w/ RNA2025.01 | 29.8 | 56.4 | 29.1 | 67 | 33.3 | 64.7 | 42.8 | 68.9 | |
| SimNPO+KLUnlearning Framework=Preference Optimization, Protocol=Original2025.01 | 29.1 | 49.9 | 27 | 61.1 | 34.7 | 60.7 | 41.7 | 64.2 | |
| DPO+MSEUnlearning Framework=Preference Optimization, Protocol=Original2025.01 | 28 | 50.2 | 19.2 | 61.1 | 28.4 | 61.5 | 36.8 | 65.9 | |
| DPO+MSEUnlearning Framework=Preference Optimization, Protocol=w/ RNA2025.01 | 27.7 | 55.6 | 23.5 | 57.5 | 30.5 | 58.4 | 39.6 | 64.3 |