Selective capability unlearning on SLURP Forget Set (DF) 1.0
95.27IFOriginal
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OriginalArchitecture=NeMo-Conformer-Transformer2026.06 | 95.27 | 91.08 | 92.64 | 90.14 | |
| OriginalArchitecture=NeMo-SSL-Conformer2026.06 | 91.05 | 84.9 | 84.42 | 83.27 | |
| NPO+KLArchitecture=NeMo-Conformer-Transformer2026.06 | 84.17 | 82.62 | 84.05 | 82.79 | |
| GA+GDArchitecture=NeMo-Conformer-Transformer2026.06 | 63.23 | 69.5 | 94.05 | 89.9 | |
| RSArchitecture=NeMo-Conformer-Transformer2026.06 | 62.7 | 71.59 | 87.01 | 88 | |
| NPO+KLArchitecture=NeMo-SSL-Conformer2026.06 | 62.7 | 58.1 | 79.81 | 76.2 | |
| RLArchitecture=NeMo-SSL-Conformer2026.06 | 61.5 | 66.28 | 75.3 | 81 | |
| RLArchitecture=NeMo-Conformer-Transformer2026.06 | 58.7 | 63.41 | 71.6 | 80.32 | |
| NPOArchitecture=NeMo-Conformer-Transformer2026.06 | 54.85 | 52.73 | 87.62 | 86.78 | |
| RSArchitecture=NeMo-SSL-Conformer2026.06 | 50.15 | 50.59 | 69.11 | 70.16 | |
| GA+GDArchitecture=NeMo-SSL-Conformer2026.06 | 48.6 | 54.2 | 71.9 | 74.13 | |
| GA+KLArchitecture=NeMo-Conformer-Transformer2026.06 | 42.6 | 44.4 | 94.78 | 88.73 | |
| NPOArchitecture=NeMo-SSL-Conformer2026.06 | 36.2 | 39.8 | 61.5 | 65.9 | |
| GA+KLArchitecture=NeMo-SSL-Conformer2026.06 | 28.9 | 31.7 | 69.4 | 71 | |
| BSUArchitecture=NeMo-Conformer-Transformer2026.06 | 28.4 | 16.22 | 22.1 | 24.8 | |
| GAArchitecture=NeMo-Conformer-Transformer2026.06 | 21.36 | 31.78 | 91.14 | 86.92 | |
| GAArchitecture=NeMo-SSL-Conformer2026.06 | 14.8 | 22.5 | 63.1 | 66.4 | |
| RetrainArchitecture=NeMo-Conformer-Transformer2026.06 | 14.04 | 20.13 | 18.59 | 23.53 | |
| RetrainArchitecture=NeMo-SSL-Conformer2026.06 | 13.72 | 16.58 | 21.38 | 22.19 | |
| BSUArchitecture=NeMo-SSL-Conformer2026.06 | 11.2 | 14.1 | 16.3 | 18.7 |