Multimodal Understanding on MMStar (5% Forget Set)
49.27Average ScoreVanilla
Evaluation results
| Method | Links | |
|---|---|---|
| VanillaBase Model=Idefics2, Forget Set Ratio=5%2025.02 | 49.27 | |
| MANUBase Model=Idefics2, Forget Set Ratio=5%2025.02 | 44.71 | |
| NPOBase Model=Idefics2, Forget Set Ratio=5%2025.02 | 42.88 | |
| KL Minim.Base Model=Idefics2, Forget Set Ratio=5%2025.02 | 41.09 | |
| Grad. DiffBase Model=Idefics2, Forget Set Ratio=5%2025.02 | 41.02 | |
| GABase Model=Idefics2, Forget Set Ratio=5%2025.02 | 40.9 | |
| VanillaBase Model=LLaVA, Forget Set Ratio=5%2025.02 | 34.8 | |
| MANUBase Model=LLaVA, Forget Set Ratio=5%2025.02 | 32.41 | |
| NPOBase Model=LLaVA, Forget Set Ratio=5%2025.02 | 32.32 | |
| KL Minim.Base Model=LLaVA, Forget Set Ratio=5%2025.02 | 31.29 | |
| Grad. DiffBase Model=LLaVA, Forget Set Ratio=5%2025.02 | 30.13 | |
| GABase Model=LLaVA, Forget Set Ratio=5%2025.02 | 29.83 |