Multimodal Language Model Unlearning on MLLMU-Bench forget set 1.0
35.14Cloze AccuracyRMU
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 35.14 | 13.01 | 0.472 | |
| POArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 28.76 | 40 | 0.245 | |
| ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 27.39 | 47.84 | 0.337 | |
| BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 25.34 | 50.54 | 0.486 | |
| KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 23.97 | 49.19 | 0.517 | |
| NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 18.49 | 50.81 | 0.472 | |
| Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 18.49 | 45.68 | 0.284 | |
| GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 17.81 | 52.16 | 0.506 | |
| RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 14.58 | 41.63 | 0.53 | |
| ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 13.69 | 39.65 | 0.465 | |
| Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 13.69 | 40.54 | 0.491 | |
| BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 13.51 | 52.43 | 0.504 | |
| GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 13.51 | 52.43 | 0.506 | |
| GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 13.51 | 51.08 | 0.472 | |
| KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 13.51 | 53.24 | 0.491 | |
| POArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 13.51 | 51.89 | 0.461 | |
| NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 13.51 | 53.24 | 0.499 | |
| ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 12.84 | 48.92 | 0.39 | |
| Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 12.84 | 50.54 | 0.359 | |
| VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 12.84 | 48.92 | 0.351 | |
| BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 12.33 | 32.42 | 0.473 | |
| GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 12.33 | 35.14 | 0.429 | |
| NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 12.33 | 32.16 | 0.485 | |
| RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 11.49 | 56.76 | 0.461 | |
| VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 10.95 | 33.51 | 0.343 | |
| GAArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 10.28 | 48.92 | 0.428 | |
| VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 10.27 | 47.83 | 0.241 | |
| POArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 10.27 | 29.46 | 0.494 | |
| GAArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 6.17 | 36.76 | 0.374 | |
| KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 4.79 | 15.68 | 0.251 |