List your company
ResearchFormerly WizWand

Multimodal Language Model Unlearning on MLLMU-Bench forget set 1.0

35.14Cloze AccuracyRMU
3.57611.770519.96528.1595May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
RMUArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0535.1413.010.472
POArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0528.76400.245
ICU-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0527.3947.840.337
BaseArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0525.3450.540.486
KL_MinArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0523.9749.190.517
NPOArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0518.4950.810.472
Noise-TArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0518.4945.680.284
GA_DiffArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0517.8152.160.506
RMUArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0514.5841.630.53
ICU-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0513.6939.650.465
Noise-TArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0513.6940.540.491
BaseArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0513.5152.430.504
GAArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0513.5152.430.506
GA_DiffArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0513.5151.080.472
KL_MinArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0513.5153.240.491
POArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0513.5151.890.461
NPOArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0513.5153.240.499
ICU-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0512.8448.920.39
Noise-TArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0512.8450.540.359
VGIDArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0512.8448.920.351
BaseArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0512.3332.420.473
GA_DiffArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0512.3335.140.429
NPOArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0512.3332.160.485
RMUArchitecture=Qwen2.5-VL-3B, Forget Ratio=15%2026.0511.4956.760.461
VGIDArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0510.9533.510.343
GAArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0510.2848.920.428
VGIDArchitecture=LLaVA-1.5-7B, Forget Ratio=15%2026.0510.2747.830.241
POArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.0510.2729.460.494
GAArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.056.1736.760.374
KL_MinArchitecture=LLaVA-1.5-13B, Forget Ratio=15%2026.054.7915.680.251