Hallucination Evaluation on CHAIR MSCOCO 2014
51.3CHAIRs Scorebaseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| baselineBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=10.232026.03 | 51.3 | 16.8 | |
| baselineBackbone=InstructBLIP, Max new tokens=512, TFLOPs=2.772026.03 | 51 | 24.2 | |
| M3IDBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=20.712026.03 | 48.3 | 13.5 | |
| ICDBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=20.632026.03 | 47.4 | 13.9 | |
| OPERABackbone=InstructBLIP, Max new tokens=5122026.03 | 47.4 | 12.9 | |
| VCDBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=20.462026.03 | 46.8 | 13.2 | |
| ICDBackbone=InstructBLIP, Max new tokens=512, TFLOPs=5.642026.03 | 46.3 | 15.3 | |
| OPERABackbone=LLaVA-1.5, Max new tokens=5122026.03 | 45.2 | 12.7 | |
| AVISCBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=20.082026.03 | 45.2 | 13.4 | |
| M3IDBackbone=InstructBLIP, Max new tokens=512, TFLOPs=5.702026.03 | 45.2 | 13.9 | |
| SIDBackbone=LLaVA-1.5, Max new tokens=5122026.03 | 44.2 | 12.2 | |
| VCDBackbone=InstructBLIP, Max new tokens=512, TFLOPs=5.612026.03 | 44 | 13.6 | |
| AVISCBackbone=InstructBLIP, Max new tokens=512, TFLOPs=5.632026.03 | 43.9 | 13.5 | |
| ProjectawayBackbone=InstructBLIP, Max new tokens=5122026.03 | 43.8 | 12.5 | |
| VTIBackbone=InstructBLIP, Max new tokens=5122026.03 | 43.4 | 11.8 | |
| SIDBackbone=InstructBLIP, Max new tokens=5122026.03 | 42.3 | 12.4 | |
| ProjectawayBackbone=LLaVA-1.5, Max new tokens=5122026.03 | 42 | 12.2 | |
| OctopusBackbone=InstructBLIP, Max new tokens=512, TFLOPs=5.872026.03 | 40.6 | 12.1 | |
| OctopusBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=21.392026.03 | 39.2 | 11.1 | |
| HIREBackbone=InstructBLIP, Max new tokens=512, TFLOPs=3.452026.03 | 39 | 11.5 | |
| VTIBackbone=LLaVA-1.5, Max new tokens=5122026.03 | 35.8 | 11.1 | |
| HIREBackbone=LLaVA-1.5, Max new tokens=512, TFLOPs=11.812026.03 | 30.2 | 9.7 | |
| baselineBackbone=InstructBLIP, Max new tokens=64, TFLOPs=1.892026.03 | 25.4 | 8.2 | |
| baselineBackbone=LLaVA-1.5, Max new tokens=64, TFLOPs=8.912026.03 | 20.4 | 6.2 | |
| NulluBackbone=LLaVA-1.5, Max new tokens=642026.03 | 17 | 5.9 | |
| HIREBackbone=LLaVA-1.5, Max new tokens=64, TFLOPs=9.182026.03 | 15.2 | 5.4 | |
| HIREBackbone=InstructBLIP, Max new tokens=64, TFLOPs=2.322026.03 | 14.8 | 5.4 | |
| M3ID+DPOBackbone=LLaVA-1.5, Max new tokens=642026.03 | 13.5 | 5.7 |