List your company
ResearchFormerly WizWand

Multimodal Language Model Unlearning on MLLMU-Bench 1.0 (test)

38.37Cloze AccuracyRMU
0.597210.403620.2130.0164May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0538.379.590.267
POArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0518.4934.590.186
NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0516.4438.110.226
BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0515.7538.380.236
ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0515.7537.570.175
Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0515.7534.860.237
VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0515.7537.290.224
KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0515.0738.920.22
GAArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0514.3835.680.203
RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0512.8452.970.297
Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0512.33400.277
VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0512.3332.970.228
BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0512.1653.780.321
ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0511.6435.140.229
GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0511.4954.60.301
KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0511.4954.320.304
NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0511.4954.050.296
RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0511.4540.410.244
GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0510.9643.510.242
GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0510.27400.241
NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0510.2739.190.264
GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0510.1454.60.316
POArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0510.1453.780.321
KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.059.5928.380.213
BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.059.5838.110.269
ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.059.4651.620.24
Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.059.4652.70.237
VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.059.4651.080.225
POArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.058.2240.270.271
GAArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.052.0536.470.199