Object Hallucination Evaluation on POPE Adversarial offline
68.96F1 ScoreBeam Search
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Beam SearchBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 68.96 | 72.87 | |
| OPERABackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 68.95 | 73.25 | |
| VASparseBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 68.88 | 73.37 | |
| VCDBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 68.63 | 72.62 | |
| SIDBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 68.3 | 72.77 | |
| HALCBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 68.04 | 72.46 | |
| GreedyBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 66.76 | 71.18 | |
| DoLaBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 66.23 | 70.8 | |
| VASparseBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 64.98 | 69.72 | |
| DoLaBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 64.71 | 69.55 | |
| GreedyBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 64.63 | 69.38 | |
| Beam SearchBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 64.29 | 69.02 | |
| SIDBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 64.1 | 69.12 | |
| VCDBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 63.77 | 69.08 | |
| HALCBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 63.51 | 68.35 | |
| OPERABackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 60.82 | 67.9 | |
| VASparseBase Model=LLaVA-1.5, Max New Token=642025.01 | 59.2 | 70.31 | |
| VASparseBase Model=LLaVA-1.52025.01 | 59.2 | — | |
| SIDBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.83 | 69.36 | |
| SIDBase Model=LLaVA-1.52025.01 | 58.83 | — | |
| HALCBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.5 | 69.25 | |
| HALCBase Model=LLaVA-1.52025.01 | 58.5 | — | |
| Beam SearchBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.43 | 69.02 | |
| Beam SearchBase Model=LLaVA-1.52025.01 | 58.43 | — | |
| LUREBase Model=LLaVA-1.5, Post-hoc=true, Max New Token=642025.01 | 58.34 | 69.16 | |
| LUREBase Model=LLaVA-1.52025.01 | 58.34 | — | |
| WoodpeckerBase Model=LLaVA-1.5, Post-hoc=true, Max New Token=642025.01 | 58.07 | 68.49 | |
| WoodpeckerBase Model=LLaVA-1.52025.01 | 58.07 | — | |
| VCDBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.02 | 68.55 | |
| VCDBase Model=LLaVA-1.52025.01 | 58.02 | — | |
| OPERABase Model=LLaVA-1.5, Max New Token=642025.01 | 58 | 68.79 | |
| OPERABase Model=LLaVA-1.52025.01 | 58 | — | |
| DoLaBase Model=LLaVA-1.5, Max New Token=642025.01 | 57.44 | 68.32 | |
| DoLaBase Model=LLaVA-1.52025.01 | 57.44 | — | |
| VASparseBase Model=mPLUG-Owl2, Max New Token=642025.01 | 56.77 | 67.86 | |
| VASparseBase Model=mPLUG-Owl22025.01 | 56.77 | — | |
| GreedyBase Model=LLaVA-1.5, Max New Token=642025.01 | 56.64 | 67.91 | |
| GreedyBase Model=LLaVA-1.52025.01 | 56.64 | — | |
| VCDBase Model=mPLUG-Owl2, Max New Token=642025.01 | 56.13 | 67.31 | |
| VCDBase Model=mPLUG-Owl22025.01 | 56.13 | — | |
| SIDBase Model=mPLUG-Owl2, Max New Token=642025.01 | 56.07 | 67.01 | |
| SIDBase Model=mPLUG-Owl22025.01 | 56.07 | — | |
| HALCBase Model=mPLUG-Owl2, Max New Token=642025.01 | 55.84 | 66.95 | |
| HALCBase Model=mPLUG-Owl22025.01 | 55.84 | — | |
| LUREBase Model=mPLUG-Owl2, Post-hoc=true, Max New Token=642025.01 | 55.65 | 66.89 | |
| LUREBase Model=mPLUG-Owl22025.01 | 55.65 | — | |
| GreedyBase Model=mPLUG-Owl2, Max New Token=642025.01 | 55.43 | 67.14 | |
| GreedyBase Model=mPLUG-Owl22025.01 | 55.43 | — | |
| WoodpeckerBase Model=mPLUG-Owl2, Post-hoc=true, Max New Token=642025.01 | 55.42 | 66.58 | |
| WoodpeckerBase Model=mPLUG-Owl22025.01 | 55.42 | — | |
| DoLaBase Model=mPLUG-Owl2, Max New Token=642025.01 | 55.24 | 67.08 | |
| DoLaBase Model=mPLUG-Owl22025.01 | 55.24 | — | |
| VASparseBackbone=MiniGPT4, Max New Tokens=5122025.01 | 54.1 | 66.02 | |
| VCDBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.98 | 65.25 | |
| GreedyBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.91 | 65.67 | |
| DoLaBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.9 | 65.84 | |
| OPERABackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.85 | 65.38 | |
| HALCBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.69 | 65.73 | |
| OPERABase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.66 | 66.43 | |
| OPERABase Model=mPLUG-Owl22025.01 | 53.66 | — | |
| SIDBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.52 | 65.45 | |
| Beam SearchBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.28 | 65.61 | |
| Beam SearchBase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.12 | 65.9 | |
| Beam SearchBase Model=mPLUG-Owl22025.01 | 53.12 | — | |
| VASparseBase Model=MiniGPT4, Max New Token=642025.01 | 52.7 | 65.91 | |
| VASparseBase Model=MiniGPT-42025.01 | 52.7 | — | |
| Beam SearchBase Model=MiniGPT4, Max New Token=642025.01 | 51.93 | 65.55 | |
| Beam SearchBase Model=MiniGPT-42025.01 | 51.93 | — | |
| GreedyBase Model=MiniGPT4, Max New Token=642025.01 | 51.92 | 65.41 | |
| GreedyBase Model=MiniGPT-42025.01 | 51.92 | — | |
| HALCBase Model=MiniGPT4, Max New Token=642025.01 | 51.8 | 65.31 | |
| HALCBase Model=MiniGPT-42025.01 | 51.8 | — | |
| SIDBase Model=MiniGPT4, Max New Token=642025.01 | 51.77 | 65.28 | |
| SIDBase Model=MiniGPT-42025.01 | 51.77 | — | |
| DoLaBase Model=MiniGPT4, Max New Token=642025.01 | 51.72 | 65.25 | |
| DoLaBase Model=MiniGPT-42025.01 | 51.72 | — | |
| LUREBase Model=MiniGPT4, Post-hoc=true, Max New Token=642025.01 | 51.38 | 65.17 | |
| LUREBase Model=MiniGPT-42025.01 | 51.38 | — | |
| WoodpeckerBase Model=MiniGPT4, Post-hoc=true, Max New Token=642025.01 | 51.27 | 65.06 | |
| WoodpeckerBase Model=MiniGPT-42025.01 | 51.27 | — | |
| OPERABase Model=MiniGPT4, Max New Token=642025.01 | 51.2 | 65.19 | |
| OPERABase Model=MiniGPT-42025.01 | 51.2 | — | |
| VCDBase Model=MiniGPT4, Max New Token=642025.01 | 51.07 | 65.02 | |
| VCDBase Model=MiniGPT-42025.01 | 51.07 | — |