List your company
ResearchFormerly WizWand

Multimodal Machine Unlearning Evaluation on MLLMU-Bench (test)

47.86Classification AccuracyVanilla
33.66437.349541.03544.7205Feb 21, 2025May 15, 2025Aug 7, 2025Oct 29, 2025Jan 21, 2026Apr 14, 2026Jul 7, 2026
Updated 3mo ago

Evaluation results

MethodLinks
VanillaBackbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1147.860.5394.8923.01
VanillaBackbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1147.530.5024.0825.33
VanillaBackbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1147.410.515.225.43
PromptingBackbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1147.290.4794.2124.11
PromptingBackbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1146.810.4833.6724.56
NPOBackbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1146.420.4084.2521.66
KL Min.Backbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1145.20.3964.5420.04
PromptingBackbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1144.870.4154.1821.99
NPOBackbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1144.440.3473.9120
MMUnlearnerBackbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1143.950.3583.8419.35
BaselineModel=LLaVA-1.5-7B, Stage=Baseline2026.0743.520.5165.225.73
Grad. Diff.Backbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1143.410.3833.8316.19
KL Min.Backbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1143.20.4393.7821.09
POPSModel=LLaVA-1.5-7B, Attacked Method=KL Minimization, Stage=POPS2026.0743.050.4513.820.8
POPSModel=LLaVA-1.5-7B, Attacked Method=Gradient Ascent, Stage=POPS2026.0742.90.4614.7218.2
KL Min.Backbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1142.750.423.2920.5
KL MinimizationModel=LLaVA-1.5-7B, Stage=Unlearned2026.0742.750.423.2920.5
SINEPROJECT(NPO)Backbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1142.670.3313.7218.81
Grad. Diff.Backbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1142.180.4013.6118.11
POPSModel=LLaVA-1.5-7B, Attacked Method=Gradient Diff, Stage=POPS2026.0741.70.4753.5517.46
MMUnlearnerBackbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1140.870.4323.3516.92
GABackbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1140.60.4213.1915.77
Gradient AscentModel=LLaVA-1.5-7B, Stage=Unlearned2026.0740.20.3873.8314.51
MMUnlearnerBackbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1140.150.4453.5217.88
MANUForget Ratio=10%, Backbone=Idefics-2-8B2025.0239.750.3553.5121.88
GABackbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1139.640.3713.717.67
NPOBackbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1139.330.4394.0117.88
Grad. Diff.Backbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1139.080.4143.0714.5
Gradient DiffModel=LLaVA-1.5-7B, Stage=Unlearned2026.0739.080.4143.0714.5
GABackbone=LLaVA-1.5-7B, Deletion Ratio=5%2025.1138.40.3843.4716.47
SINEPROJECT(NPO)Backbone=LLaVA-1.5-7B, Deletion Ratio=15%2025.1138.320.4213.0817.11
MANUForget Ratio=5%, Backbone=Idefics-2-8B2025.0235.550.3613.9120.97
MANUForget Ratio=15%, Backbone=Idefics-2-8B2025.0235.320.3333.116.66
SINEPROJECT(NPO)Backbone=LLaVA-1.5-7B, Deletion Ratio=10%2025.1134.210.4073.0119.78