Hallucination Evaluation on MSCOCO (val) (CHAIR/TPS)
23.04CHAIR_iSparseVLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SparseVLMBase Model=mPLUG-Owl2, Image Token Sparsity=true2025.01 | 23.04 | 68.42 | 23.19 | |
| FastVBase Model=mPLUG-Owl2, Image Token Sparsity=true2025.01 | 22.65 | 67.68 | 24.37 | |
| SparseVLMBase Model=MiniGPT-4, Image Token Sparsity=true2025.01 | 19.85 | 55.27 | 37.49 | |
| FastVBase Model=MiniGPT-4, Image Token Sparsity=true2025.01 | 19.67 | 54.59 | 38.05 | |
| VCDBase Model=mPLUG-Owl22025.01 | 19.46 | 62.44 | 9.77 | |
| GreedyBase Model=mPLUG-Owl22025.01 | 19.07 | 61.28 | 19.96 | |
| SIDBase Model=mPLUG-Owl22025.01 | 18.47 | 60.82 | 12.85 | |
| DoLaBase Model=mPLUG-Owl22025.01 | 18.43 | 60.18 | 14.23 | |
| WoodpeckerBase Model=mPLUG-Owl2, Post-hoc=true2025.01 | 18.39 | 59.58 | — | |
| SparseVLMBase Model=LLaVA-1.5, Image Token Sparsity=true2025.01 | 18.09 | 52.4 | 32.06 | |
| OPERABase Model=mPLUG-Owl22025.01 | 17.86 | 56.29 | 3.49 | |
| LUREBase Model=mPLUG-Owl2, Post-hoc=true2025.01 | 17.85 | 57.73 | — | |
| HALCBase Model=mPLUG-Owl22025.01 | 17.63 | 56.12 | 1.5 | |
| FastVBase Model=LLaVA-1.5, Image Token Sparsity=true2025.01 | 17.62 | 51.94 | 33.18 | |
| Beam SearchBase Model=mPLUG-Owl22025.01 | 17.12 | 54.66 | 19.58 | |
| VCDBase Model=MiniGPT-42025.01 | 17.09 | 43.8 | 17.68 | |
| VASparseBase Model=mPLUG-Owl22025.01 | 17.02 | 53.7 | 17.86 | |
| OPERABase Model=MiniGPT-42025.01 | 15.42 | 42.42 | 5.27 | |
| VCDBase Model=LLaVA-1.52025.01 | 14.82 | 49.76 | 17.55 | |
| GreedyBase Model=LLaVA-1.52025.01 | 14.63 | 49.66 | 31.17 | |
| GreedyBase Model=MiniGPT-42025.01 | 14.06 | 43.65 | 36.28 | |
| Beam SearchBase Model=MiniGPT-42025.01 | 13.9 | 44.45 | 32.1 | |
| DoLaBase Model=MiniGPT-42025.01 | 13.85 | 44.2 | 24.75 | |
| WoodpeckerBase Model=MiniGPT-4, Post-hoc=true2025.01 | 13.76 | 44.07 | — | |
| DoLaBase Model=LLaVA-1.52025.01 | 13.75 | 50.03 | 23.4 | |
| HALCBase Model=MiniGPT-42025.01 | 13.73 | 43.68 | 3.68 | |
| SIDBase Model=MiniGPT-42025.01 | 13.68 | 43.65 | 22.67 | |
| Beam SearchBase Model=LLaVA-1.52025.01 | 13.62 | 48.89 | 29.89 | |
| LUREBase Model=MiniGPT-4, Post-hoc=true2025.01 | 13.49 | 43.92 | — | |
| SIDBase Model=LLaVA-1.52025.01 | 13.29 | 47.09 | 19.57 | |
| VASparseBase Model=MiniGPT-42025.01 | 13.29 | 43.02 | 29.74 | |
| WoodpeckerBase Model=LLaVA-1.5, Post-hoc=true2025.01 | 13.27 | 49.72 | — | |
| LUREBase Model=LLaVA-1.5, Post-hoc=true2025.01 | 13.08 | 47.95 | — | |
| OPERABase Model=LLaVA-1.52025.01 | 12.98 | 47.6 | 4.07 | |
| HALCBase Model=LLaVA-1.52025.01 | 12.93 | 46.35 | 2.04 | |
| VASparseBase Model=LLaVA-1.52025.01 | 12.46 | 46.21 | 27.53 |