Visual Question Answering on MMVP (Faithfulness Evaluation)
0.8052Sentence Faithfulness (Insertion)EAGLE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| EAGLEMLLM=Qwen2.5-VL 3B [4], GPU Memory=8.76 GB, Time Consumption=222.7 s2025.09 | 0.8052 | 0.7338 | 0.8339 | 0.6634 | 0.4935 | 0.7689 | |
| EAGLEMLLM=InternVL3.5 4B [33], GPU Memory=12.26 GB, Time Consumption=176.5 s2025.09 | 0.8012 | 0.6782 | 0.8471 | 0.6379 | 0.4027 | 0.7762 | |
| EAGLEMLLM=LLaVA-1.5 7B [24], GPU Memory=15.40 GB, Time Consumption=252.6 s2025.09 | 0.796 | 0.7474 | 0.8086 | 0.6867 | 0.5027 | 0.7507 | |
| EAGLEMLLM=Qwen2.5-VL 7B [4], GPU Memory=17.40 GB, Time Consumption=220.8 s2025.09 | 0.7824 | 0.6996 | 0.8119 | 0.5901 | 0.3675 | 0.7362 | |
| LLaVA-CAMMLLM=LLaVA-1.5 7B [24], GPU Memory=34.38 GB, Time Consumption=12.2 s2025.09 | 0.7756 | 0.7745 | 0.798 | 0.6076 | 0.6044 | 0.7275 | |
| LLaVA-CAMMLLM=Qwen2.5-VL 3B [4], GPU Memory=19.17 GB, Time Consumption=16.1 s2025.09 | 0.7742 | 0.777 | 0.8181 | 0.5925 | 0.6006 | 0.7476 | |
| iGOS++ (w/ GNC)MLLM=Qwen2.5-VL 3B [4], GPU Memory=19.79 GB, Time Consumption=31.8 s2025.09 | 0.7719 | 0.7613 | 0.8183 | 0.5719 | 0.5356 | 0.7437 | |
| iGOS++ (w/ GNC)MLLM=LLaVA-1.5 7B [24], GPU Memory=92.90 GB, Time Consumption=34.8 s2025.09 | 0.7717 | 0.7698 | 0.7965 | 0.5825 | 0.5781 | 0.7236 | |
| LLaVA-CAMMLLM=Qwen2.5-VL 7B [4], GPU Memory=37.54 GB, Time Consumption=14.5 s2025.09 | 0.7505 | 0.7486 | 0.8042 | 0.4974 | 0.4847 | 0.7242 | |
| iGOS++ (w/ GNC)MLLM=Qwen2.5-VL 7B [4], GPU Memory=32.76 GB, Time Consumption=31.0 s2025.09 | 0.7394 | 0.7211 | 0.8036 | 0.4505 | 0.3853 | 0.7185 | |
| LLaVA-CAMMLLM=InternVL3.5 4B [33], GPU Memory=27.20 GB, Time Consumption=20.7 s2025.09 | 0.7348 | 0.7458 | 0.8325 | 0.4897 | 0.5213 | 0.7575 | |
| iGOS++ (w/ GNC)MLLM=InternVL3.5 4B [33], GPU Memory=62.31 GB, Time Consumption=34.4 s2025.09 | 0.7277 | 0.716 | 0.8302 | 0.4743 | 0.4454 | 0.7535 |