MLLMU-Bench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MLLMU-Bench Real-world 1.0
78.3Accuracy
16
MLLMU-Bench Real-world 1.0
77.3Accuracy
16
MLLMU-Bench 1.0 (Retain)
58.6Accuracy
16
MLLMU-Bench Retain 1.0
56Accuracy
16
MLLMU-Bench Forget 1.0
55Accuracy
16
MLLMU-Bench Forget 1.0
29.8Accuracy
16
MLLMU-Bench public celebrities Task A
98Forget Quality
9
MLLMU-Bench (retain set)
46.35Accuracy
7
MLLMU-Bench 15% forget ratio (test)
49.33Classification Forget Quality
7
MLLMU-Bench 10% forget ratio (test)
56.67Forget Quality (Class - Forget)
7
MLLMU-Bench Public Celebrities Task B
1Forget Quality
6
MLLMU-Bench (test)
42.9Test Accuracy
4
MLLMU-Bench public celebrities Task C
1Forget Quality
3
MLLMU-Bench Real Celebrity
51.8Accuracy
2