Multimodal Machine Unlearning research benchmarks

Benchmarks

Dataset NameSOTA methodMetricTrendResultsLast Updated

Retain Set

SINEPROJECT(NPO)

48.13Classification Accuracy

Mar 18, 2026

MLLMU-Bench LLaVA-1.5-7B (test 2)

62.8Forget Rate

Feb 26, 2026

MLLMU-Bench LLaVA-1.5-7B (test 1)

MMU*

65.4Forget Rate

Feb 26, 2026

SAFEERASER Model Utility v1.0 (general set)

Vanilla

67Specificity

Feb 26, 2026

SAFEERASER Generality v1.0 (test)

Vanilla

64.5ASR

Feb 26, 2026

SAFEERASER Efficacy v1.0 (forget set)

Vanilla

64.1ASR

Feb 26, 2026

MLLMU-Bench

CVF

29.6Forget VQA Accuracy

May 8, 2026

CLEAR

CVF

32.4Forget VQA Accuracy

May 8, 2026

UMU-Bench

CVF

40.4Forget VQA Acc

May 8, 2026

Forget-Set 15% forgetting ratio

KL_Min

61.45Class. VQA Accuracy

May 12, 2026

Real Celebrity

Vanilla

68.89Classification Accuracy

Feb 26, 2026

5% Forget Set (test)

Vanilla

36.8Class Acc

Feb 26, 2026

Forget Set (5%)

Vanilla

44.4Classification Accuracy

May 12, 2026