Object Hallucination Evaluation on POPE Popular offline
84.43F1 ScoreOPERA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OPERABackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 84.43 | 70.32 | |
| VASparseBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 82.44 | 72.19 | |
| HALCBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 82.4 | 70.24 | |
| SIDBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 81.9 | 71.79 | |
| Beam SearchBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 81.75 | 71.83 | |
| DoLaBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 81.72 | 71.67 | |
| GreedyBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 81.48 | 71.61 | |
| VCDBackbone=mPLUG-Owl2, Max New Tokens=5122025.01 | 80.63 | 70.67 | |
| Beam SearchBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 70.74 | 75.06 | |
| VASparseBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 70.51 | 75.16 | |
| VCDBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 70.19 | 74.56 | |
| OPERABackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 69.81 | 74.31 | |
| SIDBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 69.34 | 74.89 | |
| HALCBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 69.09 | 74.15 | |
| GreedyBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 67.99 | 72.74 | |
| DoLaBackbone=LLaVA-1.5, Max New Tokens=5122025.01 | 67.26 | 72.12 | |
| VASparseBase Model=LLaVA-1.5, Max New Token=642025.01 | 60.93 | 70.18 | |
| VASparseBase Model=LLaVA-1.52025.01 | 60.93 | — | |
| SIDBase Model=LLaVA-1.5, Max New Token=642025.01 | 59.62 | 69.42 | |
| SIDBase Model=LLaVA-1.52025.01 | 59.62 | — | |
| HALCBase Model=LLaVA-1.5, Max New Token=642025.01 | 59.33 | 69.17 | |
| HALCBase Model=LLaVA-1.52025.01 | 59.33 | — | |
| Beam SearchBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.98 | 69.31 | |
| Beam SearchBase Model=LLaVA-1.52025.01 | 58.98 | — | |
| LUREBase Model=LLaVA-1.5, Post-hoc=true, Max New Token=642025.01 | 58.63 | 69.17 | |
| LUREBase Model=LLaVA-1.52025.01 | 58.63 | — | |
| WoodpeckerBase Model=LLaVA-1.5, Post-hoc=true, Max New Token=642025.01 | 58.53 | 68.62 | |
| WoodpeckerBase Model=LLaVA-1.52025.01 | 58.53 | — | |
| OPERABase Model=LLaVA-1.5, Max New Token=642025.01 | 58.42 | 69.31 | |
| OPERABase Model=LLaVA-1.52025.01 | 58.42 | — | |
| VCDBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.34 | 68.96 | |
| VCDBase Model=LLaVA-1.52025.01 | 58.34 | — | |
| DoLaBase Model=LLaVA-1.5, Max New Token=642025.01 | 58.08 | 69.14 | |
| DoLaBase Model=LLaVA-1.52025.01 | 58.08 | — | |
| GreedyBase Model=LLaVA-1.5, Max New Token=642025.01 | 57.42 | 68.93 | |
| GreedyBase Model=LLaVA-1.52025.01 | 57.42 | — | |
| VASparseBackbone=MiniGPT4, Max New Tokens=5122025.01 | 55.68 | 66.31 | |
| VASparseBase Model=mPLUG-Owl2, Max New Token=642025.01 | 55.28 | 68.7 | |
| VASparseBase Model=mPLUG-Owl22025.01 | 55.28 | — | |
| OPERABackbone=MiniGPT4, Max New Tokens=5122025.01 | 55.04 | 66.9 | |
| VCDBase Model=mPLUG-Owl2, Max New Token=642025.01 | 54.87 | 68.55 | |
| VCDBase Model=mPLUG-Owl22025.01 | 54.87 | — | |
| DoLaBackbone=MiniGPT4, Max New Tokens=5122025.01 | 54.07 | 66.08 | |
| GreedyBackbone=MiniGPT4, Max New Tokens=5122025.01 | 54.04 | 65.84 | |
| VCDBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.89 | 65.14 | |
| HALCBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.63 | 65.62 | |
| SIDBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.53 | 65.58 | |
| SIDBase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.46 | 67.8 | |
| SIDBase Model=mPLUG-Owl22025.01 | 53.46 | — | |
| GreedyBase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.43 | 68.02 | |
| GreedyBase Model=mPLUG-Owl22025.01 | 53.43 | — | |
| OPERABase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.41 | 67.37 | |
| OPERABase Model=mPLUG-Owl22025.01 | 53.41 | — | |
| DoLaBase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.38 | 67.9 | |
| HALCBase Model=mPLUG-Owl2, Max New Token=642025.01 | 53.38 | 67.67 | |
| DoLaBase Model=mPLUG-Owl22025.01 | 53.38 | — | |
| HALCBase Model=mPLUG-Owl22025.01 | 53.38 | — | |
| Beam SearchBackbone=MiniGPT4, Max New Tokens=5122025.01 | 53.32 | 65.67 | |
| LUREBase Model=mPLUG-Owl2, Post-hoc=true, Max New Token=642025.01 | 53.15 | 67.35 | |
| LUREBase Model=mPLUG-Owl22025.01 | 53.15 | — | |
| WoodpeckerBase Model=mPLUG-Owl2, Post-hoc=true, Max New Token=642025.01 | 53.07 | 67.28 | |
| WoodpeckerBase Model=mPLUG-Owl22025.01 | 53.07 | — | |
| VASparseBase Model=MiniGPT4, Max New Token=642025.01 | 52.93 | 66.23 | |
| VASparseBase Model=MiniGPT-42025.01 | 52.93 | — | |
| Beam SearchBase Model=mPLUG-Owl2, Max New Token=642025.01 | 52.89 | 66.92 | |
| Beam SearchBase Model=mPLUG-Owl22025.01 | 52.89 | — | |
| LUREBase Model=MiniGPT4, Post-hoc=true, Max New Token=642025.01 | 52.37 | 65.96 | |
| LUREBase Model=MiniGPT-42025.01 | 52.37 | — | |
| Beam SearchBase Model=MiniGPT4, Max New Token=642025.01 | 52.27 | 66.02 | |
| Beam SearchBase Model=MiniGPT-42025.01 | 52.27 | — | |
| HALCBase Model=MiniGPT4, Max New Token=642025.01 | 52.06 | 65.83 | |
| HALCBase Model=MiniGPT-42025.01 | 52.06 | — | |
| SIDBase Model=MiniGPT4, Max New Token=642025.01 | 51.98 | 65.69 | |
| SIDBase Model=MiniGPT-42025.01 | 51.98 | — | |
| DoLaBase Model=MiniGPT4, Max New Token=642025.01 | 51.93 | 65.55 | |
| DoLaBase Model=MiniGPT-42025.01 | 51.93 | — | |
| WoodpeckerBase Model=MiniGPT4, Post-hoc=true, Max New Token=642025.01 | 51.7 | 65.49 | |
| WoodpeckerBase Model=MiniGPT-42025.01 | 51.7 | — | |
| GreedyBase Model=MiniGPT4, Max New Token=642025.01 | 51.68 | 65.31 | |
| GreedyBase Model=MiniGPT-42025.01 | 51.68 | — | |
| VCDBase Model=MiniGPT4, Max New Token=642025.01 | 51.5 | 65.61 | |
| VCDBase Model=MiniGPT-42025.01 | 51.5 | — | |
| OPERABase Model=MiniGPT4, Max New Token=642025.01 | 51.32 | 65.37 | |
| OPERABase Model=MiniGPT-42025.01 | 51.32 | — |