MLLM Unlearning on MLLMU-Bench Forget Set (10% Ratio)
40Cloze AccuracyRMU
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 40 | 12.5 | 0.52 | |
| POArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 22.92 | 44.08 | 0.403 | |
| BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 20.83 | 50.61 | 0.518 | |
| ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 20.83 | 43.27 | 0.411 | |
| Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 20.83 | 44.89 | 0.366 | |
| KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 19.79 | 48.98 | 0.518 | |
| VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 19.79 | 43.67 | 0.276 | |
| NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 18.75 | 49.79 | 0.508 | |
| GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 15.63 | 39.59 | 0.525 | |
| BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 65.31 | 0.48 | |
| GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 66.12 | 0.499 | |
| GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 66.94 | 0.455 | |
| KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 65.31 | 0.478 | |
| POArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 66.94 | 0.465 | |
| NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 65.71 | 0.486 | |
| VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 15.31 | 66.12 | 0.303 | |
| RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 14.29 | 69.39 | 0.495 | |
| ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 14.29 | 65.71 | 0.297 | |
| NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 13.54 | 34.69 | 0.506 | |
| Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=10%2026.05 | 13.27 | 68.57 | 0.296 | |
| GAArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 12.5 | 47.35 | 0.485 | |
| BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 12.5 | 34.69 | 0.489 | |
| RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=10%2026.05 | 11.49 | 56.76 | 0.461 | |
| KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 11.45 | 35.1 | 0.521 | |
| ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 11.45 | 33.87 | 0.454 | |
| VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 11.45 | 24.89 | 0.351 | |
| Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 11.42 | 35.51 | 0.407 | |
| GAArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 9.38 | 31.84 | 0.343 | |
| GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 7.29 | 29.79 | 0.512 | |
| POArchitecture=LLaVA-1.5-13B, Forget Ratio=10%2026.05 | 7.29 | 35.92 | 0.514 |