# Multimodal Machine Unlearning Evaluation on MLLMU-Bench Real Celebrity Source: [Multimodal Machine Unlearning Evaluation on MLLMU-Bench Real Celebrity]() Collection index: [llms.txt]() Data updated: 2026-03-18T07:54:57.773Z Evaluation of multimodal machine unlearning performance on the Real Celebrity subset of the MLLMU-Bench dataset, covering classification accuracy, ROUGE score, factuality score, and cloze accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Machine Unlearning Evaluation | | Dataset (including any stated split) | MLLMU-Bench Real Celebrity | | Metric | Class Acc | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Class Acc](): Higher is better; unit: PERCENT. - [ROUGE Score](): Higher is better; unit: SCALAR. - [Factuality Score](): Higher is better; unit: SCALAR. - [Cloze Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Class Acc | Result date | Report paper | | --- | --- | --- | --- | | SINEPROJECT(NPO) | 56.41 | Not recorded | 69253811d6b0dd334292f42f | | Vanilla | 52.75 | Not recorded | 68fa7a251e209131cea81495 | | Vanilla | 51.8 | Not recorded | 69253811d6b0dd334292f42f | | Vanilla | 51.8 | Not recorded | 69253811d6b0dd334292f42f | | Vanilla | 51.8 | Not recorded | 69253811d6b0dd334292f42f | | SINEPROJECT(NPO) | 51.74 | Not recorded | 69253811d6b0dd334292f42f | | Prompting | 51.6 | Not recorded | 69253811d6b0dd334292f42f | | Prompting | 51.6 | Not recorded | 69253811d6b0dd334292f42f | | Prompting | 51.6 | Not recorded | 69253811d6b0dd334292f42f | | MANU | 50.81 | Not recorded | 68fa7a251e209131cea81495 | | SINEPROJECT(NPO) | 50.77 | Not recorded | 69253811d6b0dd334292f42f | | MMUnlearner | 50.28 | Not recorded | 69253811d6b0dd334292f42f | | MANU | 50.09 | Not recorded | 68fa7a251e209131cea81495 | | NPO | 49.51 | Not recorded | 69253811d6b0dd334292f42f | | MANU | 49.31 | Not recorded | 68fa7a251e209131cea81495 | | NPO | 48.09 | Not recorded | 69253811d6b0dd334292f42f | | NPO | 47.89 | Not recorded | 69253811d6b0dd334292f42f | | MMUnlearner | 47.26 | Not recorded | 69253811d6b0dd334292f42f | | Grad. Diff. | 46.52 | Not recorded | 69253811d6b0dd334292f42f | | MMUnlearner | 45.82 | Not recorded | 69253811d6b0dd334292f42f | | KL Min. | 45.64 | Not recorded | 69253811d6b0dd334292f42f | | KL Min. | 45.58 | Not recorded | 69253811d6b0dd334292f42f | | GA | 45.56 | Not recorded | 69253811d6b0dd334292f42f | | GA | 42.64 | Not recorded | 69253811d6b0dd334292f42f | | KL Min. | 42.58 | Not recorded | 69253811d6b0dd334292f42f | | Grad. Diff. | 41.21 | Not recorded | 69253811d6b0dd334292f42f | | Grad. Diff. | 40.94 | Not recorded | 69253811d6b0dd334292f42f | | GA | 40.36 | Not recorded | 69253811d6b0dd334292f42f | ### SINEPROJECT(NPO) Class Acc: 56.41. Result ID: 69ba5a3747f34ce924a3fd74. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### Vanilla Class Acc: 52.75. Result ID: 698b6e5f7a967eddb76e88e6. [Reporting paper]() Recorded evaluation configuration: {"Forget Ratio":"5%","Backbone":"Idefics-2-8B"}. This record does not assert independent reproduction. ### Vanilla Class Acc: 51.8. Result ID: 69ba5a3747f34ce924a3fd65. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### Vanilla Class Acc: 51.8. Result ID: 69ba5a3747f34ce924a3fd6d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### Vanilla Class Acc: 51.8. Result ID: 69ba5a3747f34ce924a3fd75. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### SINEPROJECT(NPO) Class Acc: 51.74. Result ID: 69ba5a3747f34ce924a3fd6c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### Prompting Class Acc: 51.6. Result ID: 69ba5a3747f34ce924a3fd69. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### Prompting Class Acc: 51.6. Result ID: 69ba5a3747f34ce924a3fd71. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### Prompting Class Acc: 51.6. Result ID: 69ba5a3747f34ce924a3fd79. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### MANU Class Acc: 50.81. Result ID: 698b6e5f7a967eddb76e88e9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Forget Ratio":"15%","Backbone":"Idefics-2-8B"}. This record does not assert independent reproduction. ### SINEPROJECT(NPO) Class Acc: 50.77. Result ID: 69ba5a3747f34ce924a3fd7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### MMUnlearner Class Acc: 50.28. Result ID: 69ba5a3747f34ce924a3fd6b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### MANU Class Acc: 50.09. Result ID: 698b6e5f7a967eddb76e88e7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Forget Ratio":"5%","Backbone":"Idefics-2-8B"}. This record does not assert independent reproduction. ### NPO Class Acc: 49.51. Result ID: 69ba5a3747f34ce924a3fd6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### MANU Class Acc: 49.31. Result ID: 698b6e5f7a967eddb76e88e8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Forget Ratio":"10%","Backbone":"Idefics-2-8B"}. This record does not assert independent reproduction. ### NPO Class Acc: 48.09. Result ID: 69ba5a3747f34ce924a3fd7a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### NPO Class Acc: 47.89. Result ID: 69ba5a3747f34ce924a3fd72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### MMUnlearner Class Acc: 47.26. Result ID: 69ba5a3747f34ce924a3fd73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### Grad. Diff. Class Acc: 46.52. Result ID: 69ba5a3747f34ce924a3fd67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### MMUnlearner Class Acc: 45.82. Result ID: 69ba5a3747f34ce924a3fd7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### KL Min. Class Acc: 45.64. Result ID: 69ba5a3747f34ce924a3fd68. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### KL Min. Class Acc: 45.58. Result ID: 69ba5a3747f34ce924a3fd70. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### GA Class Acc: 45.56. Result ID: 69ba5a3747f34ce924a3fd66. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"5%"}. This record does not assert independent reproduction. ### GA Class Acc: 42.64. Result ID: 69ba5a3747f34ce924a3fd6e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### KL Min. Class Acc: 42.58. Result ID: 69ba5a3747f34ce924a3fd78. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### Grad. Diff. Class Acc: 41.21. Result ID: 69ba5a3747f34ce924a3fd77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. ### Grad. Diff. Class Acc: 40.94. Result ID: 69ba5a3747f34ce924a3fd6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"10%"}. This record does not assert independent reproduction. ### GA Class Acc: 40.36. Result ID: 69ba5a3747f34ce924a3fd76. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaVA-1.5-7B","Deletion Ratio":"15%"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.