Object Hallucination Assessment in Image Captioning on MSCOCO 2014 (val)
59CS ScoreBeam Search
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Beam SearchModel=Shikra-7B, Decoding Strategy=Beam Search, Max new tokens=5122026.05 | 59 | 16.3 | 74.6 | |
| VCDModel=LLaVA-1.5-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 58.6 | 18.2 | 72.8 | |
| GreedyModel=Shikra-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 57.6 | 15.7 | 75.3 | |
| VCDModel=Shikra-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 56.4 | 15.5 | 75.2 | |
| Beam SearchModel=LLaVA-1.5-7B, Decoding Strategy=Beam Search, Max new tokens=5122026.05 | 55.6 | 15.4 | 77.5 | |
| VCDModel=LLaVA-1.5-13B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 53.6 | 15.3 | 75.8 | |
| GreedyModel=LLaVA-1.5-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 53 | 15.6 | 76.7 | |
| Beam SearchModel=LLaVA-1.5-13B, Decoding Strategy=Beam Search, Max new tokens=5122026.05 | 50.4 | 13.8 | 79 | |
| GreedyModel=LLaVA-1.5-13B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 49.8 | 14.6 | 78.2 | |
| OPERAModel=LLaVA-1.5-7B, Max new tokens=5122026.05 | 45.6 | 13.1 | 79.1 | |
| OPERAModel=LLaVA-1.5-13B, Max new tokens=5122026.05 | 42.6 | 13.2 | 77.8 | |
| OPERAModel=Shikra-7B, Max new tokens=5122026.05 | 41.4 | 13.7 | 73.5 | |
| VCDModel=MiniGPT-4-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 41.4 | 14.1 | 68.2 | |
| PAIModel=Shikra-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 38.6 | 10.1 | 76.2 | |
| PAIModel=LLaVA-1.5-13B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 33 | 9.2 | 77.8 | |
| GreedyModel=MiniGPT-4-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 31.8 | 12 | 71.1 | |
| Beam SearchModel=MiniGPT-4-7B, Decoding Strategy=Beam Search, Max new tokens=5122026.05 | 29.2 | 9.9 | 71.2 | |
| DevilsModel=LLaVA-1.5-13B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 25.8 | 8.8 | 77.3 | |
| OPERAModel=MiniGPT-4-7B, Max new tokens=5122026.05 | 25.4 | 9.6 | 71.2 | |
| DevilsModel=LLaVA-1.5-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 25 | 6.7 | 76.1 | |
| PAIModel=LLaVA-1.5-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 24.2 | 7.1 | 75.2 | |
| DevilsModel=Shikra-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 23.8 | 9.4 | 72.7 | |
| PAIModel=MiniGPT-4-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 23.2 | 8.2 | 71.4 | |
| DevilsModel=MiniGPT-4-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 21.4 | 8 | 70.8 | |
| SADIModel=LLaVA-1.5-13B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 21.2 | 5.9 | 77.5 | |
| SADIModel=LLaVA-1.5-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 20.4 | 4.9 | 76.4 | |
| SADIModel=Shikra-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 20 | 5.9 | 74.8 | |
| SADIModel=MiniGPT-4-7B, Decoding Strategy=Greedy, Max new tokens=5122026.05 | 19.2 | 5.7 | 70.4 |