MLLM Unlearning on MLLMU-Bench Retain Set 10% ratio
34.4Cloze AccuracyRMU
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 34.4 | 12 | 0.287 | |
| GAArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 29.29 | 50.35 | 0.485 | |
| Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 24.02 | 27.4 | 0.416 | |
| VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 23.89 | 47.5 | 0.478 | |
| POArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 23.33 | 47.1 | 0.423 | |
| Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 22.32 | 50.31 | 0.468 | |
| ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 21.76 | 44.51 | 0.492 | |
| BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 21.09 | 49.55 | 0.496 | |
| KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 21.09 | 49.46 | 0.512 | |
| NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 18.53 | 47.59 | 0.495 | |
| GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 15.96 | 38.53 | 0.509 | |
| VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 13.54 | 26.29 | 0.417 | |
| NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 13.06 | 25.04 | 0.498 | |
| KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 13.05 | 24.24 | 0.506 | |
| BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 12.95 | 24.02 | 0.496 | |
| POArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 12.61 | 24.33 | 0.515 | |
| GAArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 11.27 | 26.03 | 0.339 | |
| ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 10.16 | 27.9 | 0.47 | |
| RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 7.07 | 49.9 | 0.509 | |
| BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.57 | 50.71 | 0.523 | |
| GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.57 | 50.3 | 0.487 | |
| POArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.57 | 50.91 | 0.485 | |
| GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.06 | 50.71 | 0.534 | |
| KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.06 | 50.71 | 0.531 | |
| NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 6.06 | 50.71 | 0.537 | |
| GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 4.79 | 34.64 | 0.496 | |
| RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 4.55 | 51.52 | 0.46 | |
| Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 4.27 | 39.9 | 0.33 | |
| VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 4.27 | 39 | 0.305 | |
| ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 4.02 | 38.29 | 0.319 |