Concept Erasure on 39 NLP settings (13 LLMs x 3 concepts)
0Mean Residual Leakage |DS|Obliviator
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ObliviatorSurgicality budget (DY)=≤ 1pp, Protocol=Alone2026.07 | 0 | 0.3333 | |
| ObliviatorSurgicality budget (DY)=≤ 1pp, Protocol=+MANCE2026.07 | 0 | 0.3333 | |
| ObliviatorSurgicality budget (DY)=≤ 3pp, Protocol=Alone2026.07 | 0 | 0.3333 | |
| ObliviatorSurgicality budget (DY)=≤ 3pp, Protocol=+MANCE2026.07 | 0 | 0.3333 | |
| ObliviatorSurgicality budget (DY)=≤ 5pp, Protocol=Alone2026.07 | 0 | 0.3333 | |
| ObliviatorSurgicality budget (DY)=≤ 5pp, Protocol=+MANCE2026.07 | 0 | 0.3333 | |
| ObliviatorSurgicality budget (DY)=≤ 10pp, Protocol=Alone2026.07 | 0 | 0.4872 | |
| ObliviatorSurgicality budget (DY)=≤ 10pp, Protocol=+MANCE2026.07 | 0 | 0.4872 | |
| MANCE++Surgicality budget (DY)=≤ 10pp, Protocol=+MANCE2026.07 | 0 | 1 | |
| MANCE++Surgicality budget (DY)=≤ 3pp, Protocol=+MANCE2026.07 | 0.1 | 0.9487 | |
| MANCE++Surgicality budget (DY)=≤ 5pp, Protocol=+MANCE2026.07 | 0.1 | 0.9744 | |
| IGBPSurgicality budget (DY)=≤ 10pp, Protocol=+MANCE2026.07 | 0.6 | 1 | |
| MANCE+Surgicality budget (DY)=≤ 10pp, Protocol=+MANCE2026.07 | 0.6 | 1 | |
| IGBPSurgicality budget (DY)=≤ 5pp, Protocol=+MANCE2026.07 | 0.7 | 1 | |
| MANCE+Surgicality budget (DY)=≤ 5pp, Protocol=+MANCE2026.07 | 0.7 | 1 | |
| IGBPSurgicality budget (DY)=≤ 3pp, Protocol=+MANCE2026.07 | 0.9 | 1 | |
| INLPSurgicality budget (DY)=≤ 10pp, Protocol=+MANCE2026.07 | 1 | 1 | |
| MANCE+Surgicality budget (DY)=≤ 3pp, Protocol=+MANCE2026.07 | 1 | 1 | |
| INLPSurgicality budget (DY)=≤ 5pp, Protocol=+MANCE2026.07 | 1.1 | 1 | |
| INLPSurgicality budget (DY)=≤ 3pp, Protocol=+MANCE2026.07 | 1.5 | 1 | |
| MANCE+Surgicality budget (DY)=≤ 1pp, Protocol=+MANCE2026.07 | 1.5 | 0.9744 | |
| IGBPSurgicality budget (DY)=≤ 1pp, Protocol=+MANCE2026.07 | 1.6 | 0.9744 | |
| MANCE (no preprocessing)Surgicality budget (DY)=≤ 10pp, Protocol=+MANCE2026.07 | 1.7 | 1 | |
| MANCE++Surgicality budget (DY)=≤ 1pp, Protocol=+MANCE2026.07 | 1.7 | 0.8462 | |
| INLPSurgicality budget (DY)=≤ 1pp, Protocol=+MANCE2026.07 | 1.8 | 0.9744 | |
| MANCE (no preprocessing)Surgicality budget (DY)=≤ 5pp, Protocol=+MANCE2026.07 | 2.2 | 1 | |
| MANCE (no preprocessing)Surgicality budget (DY)=≤ 3pp, Protocol=+MANCE2026.07 | 2.4 | 0.9744 | |
| MANCE (no preprocessing)Surgicality budget (DY)=≤ 1pp, Protocol=+MANCE2026.07 | 4.5 | 0.9487 | |
| IGBPSurgicality budget (DY)=≤ 1pp, Protocol=Alone2026.07 | 11.5 | 0.9744 | |
| IGBPSurgicality budget (DY)=≤ 3pp, Protocol=Alone2026.07 | 11.5 | 1 | |
| IGBPSurgicality budget (DY)=≤ 5pp, Protocol=Alone2026.07 | 11.5 | 1 | |
| IGBPSurgicality budget (DY)=≤ 10pp, Protocol=Alone2026.07 | 11.5 | 1 | |
| INLPSurgicality budget (DY)=≤ 1pp, Protocol=Alone2026.07 | 15.2 | 0.9231 | |
| INLPSurgicality budget (DY)=≤ 3pp, Protocol=Alone2026.07 | 15.6 | 0.9487 | |
| INLPSurgicality budget (DY)=≤ 5pp, Protocol=Alone2026.07 | 16 | 1 | |
| INLPSurgicality budget (DY)=≤ 10pp, Protocol=Alone2026.07 | 16 | 1 | |
| LEACESurgicality budget (DY)=≤ 3pp, Protocol=Alone2026.07 | 19 | 1 | |
| LEACESurgicality budget (DY)=≤ 5pp, Protocol=Alone2026.07 | 19 | 1 | |
| LEACESurgicality budget (DY)=≤ 10pp, Protocol=Alone2026.07 | 19 | 1 | |
| LEACESurgicality budget (DY)=≤ 1pp, Protocol=Alone2026.07 | 19.1 | 0.9744 |