# Multimodal Language Model Unlearning on MLLMU-Bench 1.0 (Retain Set) Source: [Multimodal Language Model Unlearning on MLLMU-Bench 1.0 (Retain Set)]() Collection index: [llms.txt]() Data updated: 2026-06-02T06:55:24.597Z Evaluation of Multimodal Language Model Unlearning performance on the Retain Set of MLLMU-Bench version 1.0, reporting Cloze Accuracy, Classification Accuracy, and Generation Relevance (RL). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Language Model Unlearning | | Dataset (including any stated split) | MLLMU-Bench 1.0 (Retain Set) | | Metric | Cloze Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Cloze Accuracy](): Higher is better; unit: PERCENT. - [Classification Accuracy](): Higher is better; unit: PERCENT. - [Generation Relevance (RL)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Cloze Accuracy | Result date | Report paper | | --- | --- | --- | --- | | RMU | 30 | Not recorded | 6a1e6366dbd7150506f8507d | | ICU-T | 26.96 | Not recorded | 6a1e6366dbd7150506f8507d | | KL\_Min | 26.36 | Not recorded | 6a1e6366dbd7150506f8507d | | Base | 25.65 | Not recorded | 6a1e6366dbd7150506f8507d | | Noise-T | 25.65 | Not recorded | 6a1e6366dbd7150506f8507d | | PO | 24.82 | Not recorded | 6a1e6366dbd7150506f8507d | | VGID | 24.46 | Not recorded | 6a1e6366dbd7150506f8507d | | NPO | 22.81 | Not recorded | 6a1e6366dbd7150506f8507d | | GA | 21.86 | Not recorded | 6a1e6366dbd7150506f8507d | | RMU | 20 | Not recorded | 6a1e6366dbd7150506f8507d | | GA\_Diff | 19.19 | Not recorded | 6a1e6366dbd7150506f8507d | | GA | 12.06 | Not recorded | 6a1e6366dbd7150506f8507d | | Noise-T | 11.05 | Not recorded | 6a1e6366dbd7150506f8507d | | ICU-T | 10.55 | Not recorded | 6a1e6366dbd7150506f8507d | | VGID | 10.3 | Not recorded | 6a1e6366dbd7150506f8507d | | GA\_Diff | 5.08 | Not recorded | 6a1e6366dbd7150506f8507d | | KL\_Min | 4.96 | Not recorded | 6a1e6366dbd7150506f8507d | | ICU-T | 4.86 | Not recorded | 6a1e6366dbd7150506f8507d | | GA\_Diff | 4.55 | Not recorded | 6a1e6366dbd7150506f8507d | | PO | 4.55 | Not recorded | 6a1e6366dbd7150506f8507d | | RMU | 4.55 | Not recorded | 6a1e6366dbd7150506f8507d | | PO | 4.25 | Not recorded | 6a1e6366dbd7150506f8507d | | Base | 4.04 | Not recorded | 6a1e6366dbd7150506f8507d | | NPO | 4.04 | Not recorded | 6a1e6366dbd7150506f8507d | | VGID | 4.02 | Not recorded | 6a1e6366dbd7150506f8507d | | GA | 3.54 | Not recorded | 6a1e6366dbd7150506f8507d | | KL\_Min | 3.54 | Not recorded | 6a1e6366dbd7150506f8507d | | Base | 3.31 | Not recorded | 6a1e6366dbd7150506f8507d | | NPO | 3.19 | Not recorded | 6a1e6366dbd7150506f8507d | | Noise-T | 3.19 | Not recorded | 6a1e6366dbd7150506f8507d | ### RMU Cloze Accuracy: 30. Result ID: 6a1e7e11fbc373592c180ad6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### ICU-T Cloze Accuracy: 26.96. Result ID: 6a1e7e11fbc373592c180acd. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### KL\_Min Cloze Accuracy: 26.36. Result ID: 6a1e7e11fbc373592c180ac9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### Base Cloze Accuracy: 25.65. Result ID: 6a1e7e11fbc373592c180ac6. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### Noise-T Cloze Accuracy: 25.65. Result ID: 6a1e7e11fbc373592c180ace. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### PO Cloze Accuracy: 24.82. Result ID: 6a1e7e11fbc373592c180aca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### VGID Cloze Accuracy: 24.46. Result ID: 6a1e7e11fbc373592c180acf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### NPO Cloze Accuracy: 22.81. Result ID: 6a1e7e11fbc373592c180acb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### GA Cloze Accuracy: 21.86. Result ID: 6a1e7e11fbc373592c180ac7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### RMU Cloze Accuracy: 20. Result ID: 6a1e7e11fbc373592c180acc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### GA\_Diff Cloze Accuracy: 19.19. Result ID: 6a1e7e11fbc373592c180ac8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-7B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### GA Cloze Accuracy: 12.06. Result ID: 6a1e7e11fbc373592c180ad1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### Noise-T Cloze Accuracy: 11.05. Result ID: 6a1e7e11fbc373592c180ac4. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### ICU-T Cloze Accuracy: 10.55. Result ID: 6a1e7e11fbc373592c180ac3. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### VGID Cloze Accuracy: 10.3. Result ID: 6a1e7e11fbc373592c180ac5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### GA\_Diff Cloze Accuracy: 5.08. Result ID: 6a1e7e11fbc373592c180ad2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### KL\_Min Cloze Accuracy: 4.96. Result ID: 6a1e7e11fbc373592c180ad3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### ICU-T Cloze Accuracy: 4.86. Result ID: 6a1e7e11fbc373592c180ad7. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### GA\_Diff Cloze Accuracy: 4.55. Result ID: 6a1e7e11fbc373592c180abe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### PO Cloze Accuracy: 4.55. Result ID: 6a1e7e11fbc373592c180ac0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### RMU Cloze Accuracy: 4.55. Result ID: 6a1e7e11fbc373592c180ac2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### PO Cloze Accuracy: 4.25. Result ID: 6a1e7e11fbc373592c180ad4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### Base Cloze Accuracy: 4.04. Result ID: 6a1e7e11fbc373592c180abc. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### NPO Cloze Accuracy: 4.04. Result ID: 6a1e7e11fbc373592c180ac1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### VGID Cloze Accuracy: 4.02. Result ID: 6a1e7e11fbc373592c180ad9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### GA Cloze Accuracy: 3.54. Result ID: 6a1e7e11fbc373592c180abd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### KL\_Min Cloze Accuracy: 3.54. Result ID: 6a1e7e11fbc373592c180abf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"Qwen2.5-VL-3B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### Base Cloze Accuracy: 3.31. Result ID: 6a1e7e11fbc373592c180ad0. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### NPO Cloze Accuracy: 3.19. Result ID: 6a1e7e11fbc373592c180ad5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. ### Noise-T Cloze Accuracy: 3.19. Result ID: 6a1e7e11fbc373592c180ad8. [Reporting paper]() Recorded evaluation configuration: {"Architecture":"LLaVA-1.5-13B","Forget Ratio":"15%"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.