Vision-Language Model Editing on FVQA 1.0 (test)
100AccuracyReasonEdit
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ReasonEditVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 100 | 0.97 | 0.95 | 0.97 | 0.87 | 100 | |
| GRACEVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 98 | 0.69 | 0.92 | 0.54 | 0.76 | 99 | |
| GRACE-COTVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 98 | 0.69 | 0.92 | 0.54 | 0.76 | 99 | |
| GRACEVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 98 | 0.59 | 0.91 | 0.56 | 0.8 | 99 | |
| GRACE-COTVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 98 | 0.57 | 0.91 | 0.57 | 0.8 | 98 | |
| GRACEVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 98 | 0.79 | 0.79 | 0.6 | 0.75 | 86 | |
| GRACE-COTVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 98 | 0.79 | 0.79 | 0.6 | 0.75 | 86 | |
| GRACEVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 98 | 0.77 | 0.82 | 0.6 | 0.73 | 93 | |
| GRACE-COTVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 98 | 0.77 | 0.82 | 0.6 | 0.73 | 93 | |
| ReasonEditVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 98 | 0.85 | 0.91 | 0.95 | 0.83 | 100 | |
| ReasonEditVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 96 | 0.94 | 0.92 | 0.92 | 0.88 | 100 | |
| ReasonEditVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 96 | 0.85 | 0.92 | 0.95 | 0.81 | 100 | |
| IKEVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 93 | 0.87 | 0.95 | 0.86 | 0.75 | 100 | |
| IKE-COTVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 93 | 0.85 | 0.94 | 0.84 | 0.8 | 100 | |
| IKEVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 92 | 0.8 | 0.97 | 0.7 | 0.78 | 100 | |
| IKE-COTVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 92 | 0.78 | 0.9 | 0.77 | 0.79 | 100 | |
| IKE-COTVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 91 | 0.73 | 0.87 | 0.69 | 0.78 | 100 | |
| IKEVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 90 | 0.76 | 0.86 | 0.7 | 0.75 | 100 | |
| IKE-COTVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 88 | 0.79 | 0.86 | 0.85 | 0.81 | 100 | |
| IKEVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 87 | 0.8 | 0.86 | 0.87 | 0.79 | 100 | |
| BalancEdit-COTVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 86 | 0.8 | 0.6 | 0.57 | 0.6 | 71 | |
| FT-COTVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 85 | 0.77 | 0.75 | 0.71 | 0.61 | 66 | |
| BalancEditVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 85 | 0.62 | 0.64 | 0.47 | 0.55 | 60 | |
| FT-COTVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 84 | 0.68 | 0.79 | 0.61 | 0.67 | 71 | |
| FTVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 83 | 0.76 | 0.69 | 0.7 | 0.39 | 53 | |
| BalancEditVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 83 | 0.79 | 0.53 | 0.47 | 0.47 | 49 | |
| FTVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 78 | 0.64 | 0.68 | 0.56 | 0.4 | 57 | |
| FT-COTVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 78 | 0.71 | 0.62 | 0.66 | 0.5 | 48 | |
| BalancEditVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 73 | 0.71 | 0.38 | 0.45 | 0.32 | 39 | |
| FTVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 69 | 0.64 | 0.53 | 0.58 | 0.27 | 38 | |
| FT-COTVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 68 | 0.65 | 0.57 | 0.62 | 0.49 | 49 | |
| BalancEditVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 66 | 0.64 | 0.4 | 0.41 | 0.32 | 31 | |
| FTVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 64 | 0.6 | 0.51 | 0.58 | 0.27 | 36 | |
| BalancEdit-COTVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 55 | 0.49 | 0.61 | 0.61 | 0.73 | 88 | |
| MENDVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 52 | 0.5 | 0.45 | 0.49 | 0.26 | 40 | |
| MEND-COTVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 50 | 0.44 | 0.58 | 0.45 | 0.51 | 66 | |
| BalancEdit-COTVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 46 | 0.45 | 0.43 | 0.41 | 0.57 | 52 | |
| MEND-COTVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 45 | 0.41 | 0.53 | 0.45 | 0.49 | 59 | |
| MENDVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 44 | 0.42 | 0.37 | 0.41 | 0.2 | 29 | |
| BalancEdit-COTVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 40 | 0.4 | 0.39 | 0.41 | 0.6 | 42 | |
| MEND-COTVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 38 | 0.38 | 0.37 | 0.34 | 0.34 | 38 | |
| MEND-COTVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 36 | 0.33 | 0.29 | 0.33 | 0.33 | 32 | |
| MENDVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 33 | 0.33 | 0.29 | 0.29 | 0.19 | 27 | |
| MENDVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 28 | 0.25 | 0.27 | 0.26 | 0.22 | 34 | |
| UneditedVLM Backbone=LLaVA-1.5-7B, Error Count=8112026.02 | 0 | 0.17 | 0.52 | 0.45 | 0.76 | 100 | |
| UneditedVLM Backbone=InstructBLIP-7B, Error Count=7642026.02 | 0 | 0.25 | 0.52 | 0.54 | 0.8 | 100 | |
| UneditedVLM Backbone=Qwen3-VL-8B, Error Count=8572026.02 | 0 | 0.16 | 0.47 | 0.4 | 0.74 | 100 | |
| UneditedVLM Backbone=Qwen3-VL-4B, Error Count=7002026.02 | 0 | 0.18 | 0.48 | 0.43 | 0.73 | 100 |