# Unlearning on MUSE-Books Harry Potter (forget set 500 samples) Source: [Unlearning on MUSE-Books Harry Potter (forget set 500 samples)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:59:13.820Z Evaluation of Machine Unlearning performance specifically on the 500-sample forget set subset of the MUSE-Books Harry Potter dataset, focusing on the R-Forget-500 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Unlearning | | Dataset (including any stated split) | MUSE-Books Harry Potter forget set 500 samples | | Metric | R-Forget-500 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 13 of 13 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [R-Forget-500](): Higher is better; unit: PERCENT. ## Recorded results | Method | R-Forget-500 | Result date | Report paper | | --- | --- | --- | --- | | Base Model (Llama3.2-3B) | 39.99 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA + KL (Dr) | 38.29 | Not recorded | 697c3af7a8b01e76cc8da3ee | | Refusal-Training | 37.75 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA (DQA\_f) + KL (Dr) | 36.87 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO (DQA\_f) | 34.28 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO + KL (Dr) | 33.62 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO | 26.83 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO (DQA\_f) + KL (Dr) | 25.6 | Not recorded | 697c3af7a8b01e76cc8da3ee | | SimNPO | 21.41 | Not recorded | 697c3af7a8b01e76cc8da3ee | | DUET | 5.98 | Not recorded | 697c3af7a8b01e76cc8da3ee | | FLAT | 0.64 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA | 0 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA (DQA\_f) | 0 | Not recorded | 697c3af7a8b01e76cc8da3ee | ### Base Model (Llama3.2-3B) R-Forget-500: 39.99. Result ID: 6998ffe75698665ec4652714. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content"}. This record does not assert independent reproduction. ### GA + KL (Dr) R-Forget-500: 38.29. Result ID: 6998ffe75698665ec4652716. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### Refusal-Training R-Forget-500: 37.75. Result ID: 6998ffe75698665ec465271d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"DQR\_f ∪ Dr"}. This record does not assert independent reproduction. ### GA (DQA\_f) + KL (Dr) R-Forget-500: 36.87. Result ID: 6998ffe75698665ec4652718. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### NPO (DQA\_f) R-Forget-500: 34.28. Result ID: 6998ffe75698665ec465271b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples"}. This record does not assert independent reproduction. ### NPO + KL (Dr) R-Forget-500: 33.62. Result ID: 6998ffe75698665ec465271a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### NPO R-Forget-500: 26.83. Result ID: 6998ffe75698665ec4652719. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content"}. This record does not assert independent reproduction. ### NPO (DQA\_f) + KL (Dr) R-Forget-500: 25.6. Result ID: 6998ffe75698665ec465271c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### SimNPO R-Forget-500: 21.41. Result ID: 6998ffe75698665ec465271e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content"}. This record does not assert independent reproduction. ### DUET R-Forget-500: 5.98. Result ID: 6998ffe75698665ec4652720. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Dquery\_f ∪ Dr"}. This record does not assert independent reproduction. ### FLAT R-Forget-500: 0.64. Result ID: 6998ffe75698665ec465271f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content"}. This record does not assert independent reproduction. ### GA R-Forget-500: 0. Result ID: 6998ffe75698665ec4652715. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content"}. This record does not assert independent reproduction. ### GA (DQA\_f) R-Forget-500: 0. Result ID: 6998ffe75698665ec4652717. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.