Object Hallucination in Open-ended Captioning on CHAIR (test)
62.3CHAIR_SM3ID
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| M3IDBackbone=InstructBLIP, Max Token=1282026.02 | 62.3 | 18.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WoodpeckerBackbone=InstructBLIP, Max Token=1282026.02 | 60.8 | 17.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VCDBackbone=InstructBLIP, Max Token=1282026.02 | 60.7 | 18 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WoodpeckerBackbone=LLaVA-1.5, Max Token=1282026.02 | 57.6 | 16.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=InstructBLIP, Max Token=1282026.02 | 57.4 | 17.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| M3IDBackbone=LLaVA-1.5, Max Token=1282026.02 | 56.6 | 15.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=LLaVA-1.5, Max Token=1282026.02 | 55.1 | 16.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AGLABackbone=InstructBLIP, Max Token=1282026.02 | 54.8 | 16.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VCDBackbone=LLaVA-1.5, Max Token=1282026.02 | 54.4 | 16.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OPERABackbone=InstructBLIP, Max Token=1282026.02 | 54.2 | 14.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PAIBackbone=InstructBLIP, Max Token=1282026.02 | 54.2 | 15.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HALCBackbone=InstructBLIP, Max Token=1282026.02 | 53.8 | 15.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VAFBackbone=InstructBLIP, Max Token=1282026.02 | 53.4 | 15.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PAIBackbone=LLaVA-1.5, Max Token=1282026.02 | 53.1 | 15.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AGLABackbone=LLaVA-1.5, Max Token=1282026.02 | 52.4 | 14.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VARBackbone=InstructBLIP, Max Token=1282026.02 | 52.3 | 14.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONLYBackbone=InstructBLIP, Max Token=1282026.02 | 52.2 | 15.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Qwen-VL, Max Token=1282026.02 | 52.1 | 16.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PADEBackbone=InstructBLIP, Max Token=1282026.02 | 51.8 | 14.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WoodpeckerBackbone=Qwen-VL, Max Token=1282026.02 | 51.8 | 16.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OPERABackbone=LLaVA-1.5, Max Token=1282026.02 | 51.6 | 14.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HALCBackbone=LLaVA-1.5, Max Token=1282026.02 | 51 | 14.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VCDBackbone=Qwen-VL, Max Token=1282026.02 | 50.4 | 17.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 50.4 | 14.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VAFBackbone=LLaVA-1.5, Max Token=1282026.02 | 50.1 | 14.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONLYBackbone=LLaVA-1.5, Max Token=1282026.02 | 49.8 | 14.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| M3IDBackbone=Qwen-VL, Max Token=1282026.02 | 49.8 | 17.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AGLABackbone=Qwen-VL, Max Token=1282026.02 | 49.8 | 15.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VARBackbone=LLaVA-1.5, Max Token=1282026.02 | 49.6 | 14.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HALCBackbone=Qwen-VL, Max Token=1282026.02 | 49.6 | 15.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VCDBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 49.6 | 14.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PAIBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 49.4 | 14.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PAIBackbone=Qwen-VL, Max Token=1282026.02 | 49.1 | 15.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONLYBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 48.9 | 13.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VAFBackbone=Qwen-VL, Max Token=1282026.02 | 48.7 | 14.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PADEBackbone=LLaVA-1.5, Max Token=1282026.02 | 48.6 | 13.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OPERABackbone=Qwen-VL, Max Token=1282026.02 | 48.6 | 14.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VAFBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 48.6 | 13.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VARBackbone=Qwen-VL, Max Token=1282026.02 | 48.4 | 13.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VARBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 48.2 | 13.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONLYBackbone=Qwen-VL, Max Token=1282026.02 | 48 | 14.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PADEBackbone=Qwen-VL, Max Token=1282026.02 | 47.8 | 13.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PADEBackbone=LLaVA-1.5-13B, Max Token=1282026.02 | 47.3 | 12.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=LLaVA-Next, Max Token=1282026.02 | 30.2 | 10.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VCDBackbone=LLaVA-Next, Max Token=1282026.02 | 29.8 | 10.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PAIBackbone=LLaVA-Next, Max Token=1282026.02 | 29.4 | 10.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ONLYBackbone=LLaVA-Next, Max Token=1282026.02 | 28.6 | 9.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VAFBackbone=LLaVA-Next, Max Token=1282026.02 | 28.4 | 9.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VARBackbone=LLaVA-Next, Max Token=1282026.02 | 28.2 | 9.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PADEBackbone=LLaVA-Next, Max Token=1282026.02 | 27.8 | 8.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CGDType=Training-free mitigation2026.03 | — | — | 52.4 | 51.4 | 59.1 | 56 | 53.8 | 52.4 | 54 | 56 | 55.4 | 57.2 | 56.1 | 59.8 | 52.2 | 55.6 | 51.6 | 54.9 | |
| ClipTTTLoRA rank=8, LoRA scaling factor=16, Adaptation iterations=70, Optimizer=AdamW, Learning Rate Scheduler=Cosine annealing2026.03 | — | — | 48 | 44.4 | 50.8 | 45 | 44.8 | 46.6 | 48.4 | 50.3 | 49.6 | 46 | 52.2 | 48.4 | 48.4 | 50 | 45.6 | 47.9 | |
| Greedy DecodingDescription=Standard decoding on corrupted images, Type=Baseline2026.03 | — | — | 56.8 | 51.8 | 57.8 | 54.2 | 53.8 | 52.6 | 55.4 | 55 | 55.4 | 55.8 | 58 | 58.2 | 55.4 | 57 | 56.4 | 55.6 | |
| NL-MeansType=Image pre-processing baseline2026.03 | — | — | 57.6 | 42 | 53.2 | 56.8 | 54.2 | 55 | 55 | 57.2 | 54 | 58.8 | 60.8 | 57.8 | 55 | 55.8 | 52.4 | 55 | |
| PAIType=Training-free mitigation2026.03 | — | — | 58.6 | 48.6 | 56.6 | 54.6 | 51.6 | 54 | 56.4 | 55.6 | 56.8 | 57.2 | 62.2 | 58.2 | 54 | 55.2 | 57 | 55.8 | |
| RestormerType=Image pre-processing baseline2026.03 | — | — | 56.2 | 47.4 | 57.6 | 53.6 | 53.8 | 52.8 | 57.2 | 54 | 59.4 | 56.8 | 58.6 | 57 | 57.6 | 54.2 | 55.2 | 55.4 | |
| VAPType=Training-free mitigation2026.03 | — | — | 54 | 46.8 | 56.7 | 53.9 | 50.7 | 53.1 | 52 | 50.9 | 52.3 | 55.2 | 55.9 | 57.5 | 52.6 | 49.7 | 50.2 | 52.8 | |
| VCDType=Training-free mitigation2026.03 | — | — | 55.6 | 50 | 56.6 | 51.4 | 54.2 | 53.6 | 54 | 54.8 | 54.6 | 57.6 | 58.8 | 55.8 | 54 | 55.2 | 53.2 | 54.6 |