# Unlearning on MUSE-Books Harry Potter (100 samples, forget set) Source: [Unlearning on MUSE-Books Harry Potter (100 samples, forget set)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:59:13.803Z Evaluation of Machine Unlearning performance on the specific 100-sample subset (forget set) of the MUSE-Books Harry Potter dataset, reporting R-Forget and Performance Shift. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Unlearning | | Dataset (including any stated split) | MUSE-Books Harry Potter 100 samples (forget set) | | Metric | R-Forget | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 13 of 13 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [R-Forget](): Higher is better; unit: PERCENT. - [Performance Shift](): Lower is better; unit: PERCENT. ## Recorded results | Method | R-Forget | Result date | Report paper | | --- | --- | --- | --- | | Base Model (Llama3.2-3B) | 32.13 | Not recorded | 697c3af7a8b01e76cc8da3ee | | Refusal-Training | 31.02 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO (DQA\_f) | 30.19 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO + KL (Dr) | 28.92 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA (DQA\_f) + KL (Dr) | 27.44 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA + KL (Dr) | 27.2 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO | 24.18 | Not recorded | 697c3af7a8b01e76cc8da3ee | | NPO (DQA\_f) + KL (Dr) | 21.55 | Not recorded | 697c3af7a8b01e76cc8da3ee | | SimNPO | 17.6 | Not recorded | 697c3af7a8b01e76cc8da3ee | | DUET | 4.27 | Not recorded | 697c3af7a8b01e76cc8da3ee | | FLAT | 0.47 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA | 0 | Not recorded | 697c3af7a8b01e76cc8da3ee | | GA (DQA\_f) | 0 | Not recorded | 697c3af7a8b01e76cc8da3ee | ### Base Model (Llama3.2-3B) R-Forget: 32.13. Result ID: 6998ffe75698665ec4652707. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"None"}. This record does not assert independent reproduction. ### Refusal-Training R-Forget: 31.02. Result ID: 6998ffe75698665ec4652710. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"DQR\_f ∪ Dr","Regularization":"None"}. This record does not assert independent reproduction. ### NPO (DQA\_f) R-Forget: 30.19. Result ID: 6998ffe75698665ec465270e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples","Regularization":"None"}. This record does not assert independent reproduction. ### NPO + KL (Dr) R-Forget: 28.92. Result ID: 6998ffe75698665ec465270d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### GA (DQA\_f) + KL (Dr) R-Forget: 27.44. Result ID: 6998ffe75698665ec465270b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### GA + KL (Dr) R-Forget: 27.2. Result ID: 6998ffe75698665ec4652709. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### NPO R-Forget: 24.18. Result ID: 6998ffe75698665ec465270c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"None"}. This record does not assert independent reproduction. ### NPO (DQA\_f) + KL (Dr) R-Forget: 21.55. Result ID: 6998ffe75698665ec465270f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples","Regularization":"KL-divergence"}. This record does not assert independent reproduction. ### SimNPO R-Forget: 17.6. Result ID: 6998ffe75698665ec4652711. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"None"}. This record does not assert independent reproduction. ### DUET R-Forget: 4.27. Result ID: 6998ffe75698665ec4652713. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Dquery\_f ∪ Dr","Regularization":"None"}. This record does not assert independent reproduction. ### FLAT R-Forget: 0.47. Result ID: 6998ffe75698665ec4652712. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"None"}. This record does not assert independent reproduction. ### GA R-Forget: 0. Result ID: 6998ffe75698665ec4652708. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"Raw book content","Regularization":"None"}. This record does not assert independent reproduction. ### GA (DQA\_f) R-Forget: 0. Result ID: 6998ffe75698665ec465270a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B-Instruct","Training Data Format":"QA samples","Regularization":"None"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.