Open-ended Multimodal Understanding Benchmark Average
0.799AUCCOMET
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| COMETBackbone=Qwen3VL-8B-Instruct, Generation-free (GF)=true2026.06 | 0.799 | 0.728 | 0.701 | |
| COMETBackbone=Qwen3VL-2B-Instruct, Generation-free (GF)=true2026.06 | 0.785 | 0.784 | 0.707 | |
| COMETBackbone=Qwen3VL-4B-Instruct, Generation-free (GF)=true2026.06 | 0.784 | 0.631 | 0.717 | |
| COMETBackbone=InternVL3.5-1B, Generation-free (GF)=true2026.06 | 0.734 | 0.647 | 0.771 | |
| P(Correct)Backbone=Qwen3VL-8B-Instruct, Generation-free (GF)=false2026.06 | 0.714 | 0.652 | 0.766 | |
| P(Correct)Backbone=Qwen3VL-4B-Instruct, Generation-free (GF)=false2026.06 | 0.712 | 0.536 | 0.837 | |
| P(Correct)Backbone=Qwen3VL-2B-Instruct, Generation-free (GF)=false2026.06 | 0.685 | 0.648 | 0.863 | |
| P(Correct)Backbone=InternVL3.5-1B, Generation-free (GF)=false2026.06 | 0.667 | 0.532 | 0.897 | |
| Consist. (Ent)Backbone=Qwen3VL-4B-Instruct, Generation-free (GF)=false2026.06 | 0.661 | 0.464 | 0.899 | |
| Consist. (Max)Backbone=Qwen3VL-4B-Instruct, Generation-free (GF)=false2026.06 | 0.66 | 0.464 | 0.899 | |
| Consist. (Ent)Backbone=Qwen3VL-8B-Instruct, Generation-free (GF)=false2026.06 | 0.657 | 0.548 | 0.904 | |
| Consist. (Max)Backbone=Qwen3VL-8B-Instruct, Generation-free (GF)=false2026.06 | 0.655 | 0.547 | 0.904 | |
| VL UncertaintyBackbone=Qwen3VL-4B-Instruct, Generation-free (GF)=false2026.06 | 0.654 | 0.605 | 0.829 | |
| VL UncertaintyBackbone=Qwen3VL-8B-Instruct, Generation-free (GF)=false2026.06 | 0.654 | 0.665 | 0.829 | |
| VL UncertaintyBackbone=Qwen3VL-2B-Instruct, Generation-free (GF)=false2026.06 | 0.65 | 0.646 | 0.852 | |
| Verbalized ConfBackbone=Qwen3VL-8B-Instruct, Generation-free (GF)=false2026.06 | 0.642 | 0.598 | 0.787 | |
| Verbalized ConfBackbone=InternVL3.5-1B, Generation-free (GF)=false2026.06 | 0.639 | 0.546 | 0.84 | |
| VL UncertaintyBackbone=InternVL3.5-1B, Generation-free (GF)=false2026.06 | 0.637 | 0.535 | 0.869 | |
| Consist. (Ent)Backbone=Qwen3VL-2B-Instruct, Generation-free (GF)=false2026.06 | 0.624 | 0.593 | 0.918 | |
| Consist. (Max)Backbone=Qwen3VL-2B-Instruct, Generation-free (GF)=false2026.06 | 0.623 | 0.593 | 0.914 | |
| Verbalized ConfBackbone=Qwen3VL-2B-Instruct, Generation-free (GF)=false2026.06 | 0.591 | 0.592 | 0.842 | |
| NLLBackbone=InternVL3.5-1B, Generation-free (GF)=false2026.06 | 0.576 | 0.433 | 0.949 | |
| NLLBackbone=Qwen3VL-2B-Instruct, Generation-free (GF)=false2026.06 | 0.569 | 0.545 | 0.963 | |
| Verbalized ConfBackbone=Qwen3VL-4B-Instruct, Generation-free (GF)=false2026.06 | 0.562 | 0.378 | 0.896 | |
| Consist. (Ent)Backbone=InternVL3.5-1B, Generation-free (GF)=false2026.06 | 0.495 | 0.411 | 0.959 | |
| Consist. (Max)Backbone=InternVL3.5-1B, Generation-free (GF)=false2026.06 | 0.495 | 0.411 | 0.959 | |
| NLLBackbone=Qwen3VL-4B-Instruct, Generation-free (GF)=false2026.06 | 0.491 | 0.357 | 0.974 | |
| NLLBackbone=Qwen3VL-8B-Instruct, Generation-free (GF)=false2026.06 | 0.475 | 0.417 | 0.97 |