Multimodal Language Model Unlearning on MLLMU-Bench 1.0 (test)
38.37Cloze AccuracyRMU
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 38.37 | 9.59 | 0.267 | |
| POArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 18.49 | 34.59 | 0.186 | |
| NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 16.44 | 38.11 | 0.226 | |
| BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 15.75 | 38.38 | 0.236 | |
| ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 15.75 | 37.57 | 0.175 | |
| Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 15.75 | 34.86 | 0.237 | |
| VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 15.75 | 37.29 | 0.224 | |
| KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 15.07 | 38.92 | 0.22 | |
| GAArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 14.38 | 35.68 | 0.203 | |
| RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 12.84 | 52.97 | 0.297 | |
| Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 12.33 | 40 | 0.277 | |
| VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 12.33 | 32.97 | 0.228 | |
| BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 12.16 | 53.78 | 0.321 | |
| ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 11.64 | 35.14 | 0.229 | |
| GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 11.49 | 54.6 | 0.301 | |
| KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 11.49 | 54.32 | 0.304 | |
| NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 11.49 | 54.05 | 0.296 | |
| RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 11.45 | 40.41 | 0.244 | |
| GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.05 | 10.96 | 43.51 | 0.242 | |
| GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 10.27 | 40 | 0.241 | |
| NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 10.27 | 39.19 | 0.264 | |
| GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 10.14 | 54.6 | 0.316 | |
| POArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 10.14 | 53.78 | 0.321 | |
| KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 9.59 | 28.38 | 0.213 | |
| BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 9.58 | 38.11 | 0.269 | |
| ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 9.46 | 51.62 | 0.24 | |
| Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 9.46 | 52.7 | 0.237 | |
| VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.05 | 9.46 | 51.08 | 0.225 | |
| POArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 8.22 | 40.27 | 0.271 | |
| GAArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.05 | 2.05 | 36.47 | 0.199 |