Knowledge Unlearning on The Pile 32 sample (val)
0EL10 (%)NEO + DPD+
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| NEO + DPD+Params=125M2022.10 | 0 | 27.4 | |
| NEO + DPD+Params=1.3B2022.10 | 0 | 21.4 | |
| NEO + DPD+Params=2.7B2022.10 | 0 | 24.2 | |
| NEO + UL+Params=125M, Epoch=17.22022.10 | 1 | 27.4 | |
| NEO + UL+Params=2.7B, Epoch=10.82022.10 | 1.6 | 31 | |
| NEO + UL+Params=1.3B, Epoch=13.82022.10 | 1.9 | 30.4 | |
| NEO + ULParams=125M, Epoch=112022.10 | 3.7 | 50.1 | |
| OPTParams=125M2022.10 | 8.6 | 52.9 | |
| NEO + ULParams=1.3B, Epoch=82022.10 | 11 | 62.2 | |
| NEO + ULParams=2.7B, Epoch=5.42022.10 | 13 | 66 | |
| OPTParams=1.3B2022.10 | 23.3 | 67.1 | |
| OPTParams=2.7B2022.10 | 25.6 | 69.2 | |
| NEOParams=125M2022.10 | 30.9 | 77.4 | |
| NEOParams=1.3B2022.10 | 67.6 | 92.2 | |
| NEOParams=2.7B2022.10 | 70.4 | 93.4 |