List your company
ResearchFormerly WizWand

VLM Editing on A-OKVQA 2022 (test)

100AccuracyGRACE-COT
-4235077Feb 2, 2026
Updated 8mo ago

Evaluation results

MethodLinks
GRACE-COTBackbone=Qwen3-VL-4B, Error count=5188, Reasoning-enabled=true2026.021009193806095
GRACE-COTBackbone=LLaVA-1.5-7B, Error count=7197, Reasoning-enabled=true2026.02998591626198
ReasonEditBackbone=LLaVA-1.5-7B, Error count=71972026.029997989884100
ReasonEditBackbone=InstructBLIP-7B, Error count=59542026.029997989986100
IKEBackbone=Qwen3-VL-8B, Error count=71392026.029986977558100
IKE-COTBackbone=Qwen3-VL-8B, Error count=7139, Reasoning-enabled=true2026.029983977470100
GRACE-COTBackbone=Qwen3-VL-8B, Error count=7139, Reasoning-enabled=true2026.02998889766190
IKEBackbone=InstructBLIP-7B, Error count=59542026.029892988567100
GRACEBackbone=InstructBLIP-7B, Error count=59542026.02987387626399
GRACEBackbone=Qwen3-VL-8B, Error count=71392026.02988683696087
ReasonEditBackbone=Qwen3-VL-8B, Error count=71392026.029892979579100
IKEBackbone=LLaVA-1.5-7B, Error count=71972026.029789958262100
GRACEBackbone=LLaVA-1.5-7B, Error count=71972026.02978489645898
IKE-COTBackbone=InstructBLIP-7B, Error count=5954, Reasoning-enabled=true2026.029791988577100
GRACE-COTBackbone=InstructBLIP-7B, Error count=5954, Reasoning-enabled=true2026.02977788586498
GRACEBackbone=Qwen3-VL-4B, Error count=51882026.02978584715987
IKE-COTBackbone=Qwen3-VL-4B, Error count=5188, Reasoning-enabled=true2026.029787967866100
ReasonEditBackbone=Qwen3-VL-4B, Error count=51882026.029790969677100
IKE-COTBackbone=LLaVA-1.5-7B, Error count=7197, Reasoning-enabled=true2026.029688958576100
IKEBackbone=Qwen3-VL-4B, Error count=51882026.029587957957100
BalancEditBackbone=LLaVA-1.5-7B, Error count=71972026.02736928324960
BalancEdit-COTBackbone=LLaVA-1.5-7B, Error count=7197, Reasoning-enabled=true2026.02736825295473
FT-COTBackbone=LLaVA-1.5-7B, Error count=7197, Reasoning-enabled=true2026.02716664635250
FT-COTBackbone=InstructBLIP-7B, Error count=5954, Reasoning-enabled=true2026.02695558595253
MENDBackbone=LLaVA-1.5-7B, Error count=71972026.02625859592835
FT-COTBackbone=Qwen3-VL-8B, Error count=7139, Reasoning-enabled=true2026.02615855584435
FT-COTBackbone=Qwen3-VL-4B, Error count=5188, Reasoning-enabled=true2026.02615852564135
FTBackbone=LLaVA-1.5-7B, Error count=71972026.02595553533440
BalancEditBackbone=InstructBLIP-7B, Error count=59542026.02594639354456
BalancEditBackbone=Qwen3-VL-8B, Error count=71392026.02565437474245
FTBackbone=InstructBLIP-7B, Error count=59542026.02544649473352
MENDBackbone=InstructBLIP-7B, Error count=59542026.02534748512733
MEND-COTBackbone=LLaVA-1.5-7B, Error count=7197, Reasoning-enabled=true2026.02524946474546
FTBackbone=Qwen3-VL-8B, Error count=71392026.02484544452428
MEND-COTBackbone=InstructBLIP-7B, Error count=5954, Reasoning-enabled=true2026.02464241434143
BalancEditBackbone=Qwen3-VL-4B, Error count=51882026.02413922243136
FTBackbone=Qwen3-VL-4B, Error count=51882026.02373533332230
BalancEdit-COTBackbone=InstructBLIP-7B, Error count=5954, Reasoning-enabled=true2026.02312328486292
MENDBackbone=Qwen3-VL-8B, Error count=71392026.02282829271922
BalancEdit-COTBackbone=Qwen3-VL-4B, Error count=5188, Reasoning-enabled=true2026.02272623255271
MEND-COTBackbone=Qwen3-VL-8B, Error count=7139, Reasoning-enabled=true2026.02262625262529
MENDBackbone=Qwen3-VL-4B, Error count=51882026.02252523252028
BalancEdit-COTBackbone=Qwen3-VL-8B, Error count=7139, Reasoning-enabled=true2026.02232524355563
MEND-COTBackbone=Qwen3-VL-4B, Error count=5188, Reasoning-enabled=true2026.02232422262630
UneditedBackbone=LLaVA-1.5-7B, Error count=71972026.02012103761100
UneditedBackbone=InstructBLIP-7B, Error count=59542026.02014114863100
UneditedBackbone=Qwen3-VL-8B, Error count=71392026.02012113358100
UneditedBackbone=Qwen3-VL-4B, Error count=51882026.02014153858100