MLLM Unlearning on MLLMU-Bench 10% (test)
40Cloze AccuracyRMU
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 40 | 7.29 | 0.287 | |
| GAArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 14.58 | 34.69 | 0.249 | |
| GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 13.54 | 31.84 | 0.271 | |
| RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 12.84 | 52.97 | 0.297 | |
| BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 12.5 | 37.96 | 0.211 | |
| KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 11.46 | 36.73 | 0.218 | |
| POArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 11.46 | 37.96 | 0.237 | |
| NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 11.46 | 37.55 | 0.233 | |
| ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 10.42 | 35.51 | 0.223 | |
| Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 9.38 | 39.59 | 0.231 | |
| VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 9.38 | 37.56 | 0.203 | |
| GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.12 | 46.12 | 0.366 | |
| KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 4.17 | 33.47 | 0.285 | |
| POArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 4.17 | 33.06 | 0.269 | |
| POArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 4.08 | 44.49 | 0.376 | |
| RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 4.08 | 46.94 | 0.357 | |
| BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 3.13 | 32.24 | 0.285 | |
| NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 3.13 | 31.84 | 0.286 | |
| BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 46.53 | 0.359 | |
| GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 45.31 | 0.359 | |
| KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 44.9 | 0.366 | |
| NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 45.31 | 0.36 | |
| ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 46.53 | 0.279 | |
| Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 44.08 | 0.257 | |
| VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 3.06 | 44.49 | 0.271 | |
| ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 2.25 | 33.47 | 0.289 | |
| VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 2.25 | 29.79 | 0.222 | |
| GAArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 2.08 | 31.43 | 0.343 | |
| Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 2.08 | 33.06 | 0.145 | |
| GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 1.04 | 31.42 | 0.279 |