List your company
ResearchFormerly WizWand

Classification on MLLMU-Bench Forget Set

63.33AccuracyBase
-2.13425614.86127231.856848.852328Feb 21, 2025May 8, 2025Jul 24, 2025Oct 8, 2025Dec 24, 2025Mar 10, 2026May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
BaseBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0563.33
GABackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0561.67
NPOBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0561.67
GA_DiffBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0560.83
KL_MinBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0560
POBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0560
Noise-TBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0554.17
VGIDBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0550
ICU-TBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0549.17
BaseBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0549.17
ICU-TBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0539.17
KL_MinBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0538.33
POBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0538.33
GABackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0537.5
NPOBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0537.5
RMUBackbone=Qwen2.5-VL-3B, Forget Ratio=5%2026.0536.67
GA_DiffBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0536.67
Noise-TBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0536.67
RMUBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0535.83
VGIDBackbone=LLaVA-1.5-7B, Forget Ratio=5%2026.0535
POBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0534.17
RMUBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0532.5
GABackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0531.67
BaseBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0528.33
NPOBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0528.33
Noise-TBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0527.5
ICU-TBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0525.83
KL_MinBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0525.33
VGIDBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0524.17
GA_DiffBackbone=LLaVA-1.5-13B, Forget Ratio=5%2026.0518.83
VanillaBase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.5187
VanillaBase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.517
PromptingBase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.4973
VanillaBase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.4915
PromptingBase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.4841
KL MinimizationBase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.476
NPOBase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.474
KL MinimizationBase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.468
PromptingBase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.468
NPOBase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.4561
NPOBase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.4552
KL MinimizationBase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.448
GABase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.444
GABase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.4385
Gradient DifferenceBase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.436
Gradient DifferenceBase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.4347
MANUBase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.4205
Gradient DifferenceBase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.416
MANUBase Model=LLaVA-1.5-7B, Forget Percentage=5%2025.020.4125
GABase Model=LLaVA-1.5-7B, Forget Percentage=15%2025.020.4093
MANUBase Model=LLaVA-1.5-7B, Forget Percentage=10%2025.020.3836