Vision-Language Question Answering on Pooled MLLM Suite (GQA, LLaVA-Wild, MMMU Pro, MME-Finance) (test)
5.4Expected Calibration Error (ECE)InternalInspector
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| InternalInspectorLVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 5.4 | 17 | 73.3 | 82.6 | 89.7 | 79.6 | |
| BICRLVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 5.6 | 18.2 | 71.4 | 78.1 | 87.7 | 78.9 | |
| PIKLVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 7.5 | 18 | 72.2 | 81.5 | 88.4 | 77.2 | |
| BICRLVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 8.9 | 17.4 | 72.9 | 78.9 | 90.3 | 80.1 | |
| SAPLMALVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 10.6 | 19.4 | 72.8 | 82.8 | 86.4 | 74.2 | |
| PIKLVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 10.8 | 19.9 | 68.9 | 78.2 | 87.1 | 77.3 | |
| PIKLVLM=OpenGVLab/InternVL3_5-14B-HF, n=30,5002026.05 | 10.8 | 20.2 | 70.5 | 80.3 | 86.4 | 73.8 | |
| PELVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 12.8 | 23.5 | 63 | 77.3 | 77.3 | 68.5 | |
| InternalInspectorLVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 13.8 | 22.3 | 65.4 | 77.8 | 81.4 | 71.9 | |
| CCPSLVLM=OpenGVLab/InternVL3_5-14B-HF, n=30,5002026.05 | 14.7 | 23.8 | 67.8 | 80 | 71.6 | 58.2 | |
| CCPSLVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 16.3 | 22 | 67.4 | 78.8 | 76 | 72.9 | |
| SAPLMALVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 16.5 | 23 | 67.3 | 78.8 | 82 | 72.5 | |
| SAPLMALVLM=OpenGVLab/InternVL3_5-14B-HF, n=30,5002026.05 | 16.6 | 23.3 | 69.6 | 80.8 | 76.2 | 65.4 | |
| PELVLM=OpenGVLab/InternVL3_5-14B-HF, n=30,5002026.05 | 16.7 | 25.9 | 66.6 | 80 | 60.3 | 43.4 | |
| PELVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 19.8 | 26 | 67.3 | 80.4 | 66.2 | 52.9 | |
| Self-ProbingLVLM=OpenGVLab/InternVL3_5-14B-HF, n=30,5002026.05 | 21.5 | 24.7 | 68.4 | 80.8 | 76.8 | 70.8 | |
| Self-ProbingLVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 24.4 | 27.5 | 69.7 | 81.2 | 77 | 59.5 | |
| P(True)LVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 26.2 | 30.9 | 45.7 | 34.8 | 67.9 | 54.7 | |
| CCPSLVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 28.7 | 45.6 | 53.9 | 54.5 | 66.2 | 44.9 | |
| Self-ProbingLVLM=llava-hf/llava-v1.6-vicuna-13b-hf, n=30,5012026.05 | 28.9 | 30 | 65.2 | 78.1 | 81.9 | 67.3 | |
| P(True)LVLM=OpenGVLab/InternVL3_5-14B-HF, n=30,5002026.05 | 41.2 | 41.5 | 57.2 | 70 | 78 | 59.5 | |
| P(True)LVLM=Qwen/Qwen3-VL-8B-Instruct, n=30,2412026.05 | 43.9 | 44.2 | 54.6 | 67 | 76.2 | 54.6 |