# Multimodal Judgment on MM-Vet Source: [Multimodal Judgment on MM-Vet]() Collection index: [llms.txt]() Data updated: 2026-06-02T13:58:01.209Z Evaluation of model performance on the MM-Vet dataset using Multimodal Judgment metrics, often involving pairwise comparisons. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Judgment | | Dataset (including any stated split) | MM-Vet | | Metric | Overall Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Overall Score](): Higher is better; unit: PERCENT. - [Pairwise Win Rate (w/ Tie)](): Higher is better; unit: PERCENT. - [Pairwise Win Rate (w/o Tie)](): Higher is better; unit: PERCENT. - [Batch Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Overall Score | Result date | Report paper | | --- | --- | --- | --- | | Perception-Judge-Flex | 37.4 | Not recorded | 6a1e6458dbd7150506f85a32 | | GPT-4V | 36.7 | Not recorded | 6a1e6458dbd7150506f85a32 | | Qwen3-VL-Thinking | 35.2 | Not recorded | 6a1e6458dbd7150506f85a32 | | GPT-4o | 34.2 | Not recorded | 6a1e6458dbd7150506f85a32 | | Perception-Judge-Qwen3 | 33.2 | Not recorded | 6a1e6458dbd7150506f85a32 | | Flex-Judge-VL | 33 | Not recorded | 6a1e6458dbd7150506f85a32 | | LLaVA-1.5 | 32.2 | Not recorded | 6a1e6458dbd7150506f85a32 | | Flex-Judge-VL | 31.1 | Not recorded | 6a1e6458dbd7150506f85a32 | | Perception-Judge-Flex | 31 | Not recorded | 6a1e6458dbd7150506f85a32 | | Qwen3-VL-Thinking | 30.8 | Not recorded | 6a1e6458dbd7150506f85a32 | | LLaVA-Critic | 27.2 | Not recorded | 6a1e6458dbd7150506f85a32 | | Perception-Judge-Qwen3 | 27 | Not recorded | 6a1e6458dbd7150506f85a32 | | LLaVA-1.6 | 26.5 | Not recorded | 6a1e6458dbd7150506f85a32 | | Gemini-1.0-Pro-Vision | 20.7 | Not recorded | 6a1e6458dbd7150506f85a32 | | Prometheus-V | 15.7 | Not recorded | 6a1e6458dbd7150506f85a32 | | Qwen2.5-VL-Instruct | 10.3 | Not recorded | 6a1e6458dbd7150506f85a32 | ### Perception-Judge-Flex Overall Score: 37.4. Result ID: 6a1ee13efbc373592c1b22b0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### GPT-4V Overall Score: 36.7. Result ID: 6a1ee13efbc373592c1b22a4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LoRA Training":"true"}. This record does not assert independent reproduction. ### Qwen3-VL-Thinking Overall Score: 35.2. Result ID: 6a1ee13efbc373592c1b22af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. ### GPT-4o Overall Score: 34.2. Result ID: 6a1ee13efbc373592c1b22a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LoRA Training":"true"}. This record does not assert independent reproduction. ### Perception-Judge-Qwen3 Overall Score: 33.2. Result ID: 6a1ee13efbc373592c1b22b2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"4B"}. This record does not assert independent reproduction. ### Flex-Judge-VL Overall Score: 33. Result ID: 6a1ee13efbc373592c1b22ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"32B","LoRA Training":"true"}. This record does not assert independent reproduction. ### LLaVA-1.5 Overall Score: 32.2. Result ID: 6a1ee13efbc373592c1b22a8. [Reporting paper]() Recorded evaluation configuration: {"Size":"13B"}. This record does not assert independent reproduction. ### Flex-Judge-VL Overall Score: 31.1. Result ID: 6a1ee13efbc373592c1b22ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Perception-Judge-Flex Overall Score: 31. Result ID: 6a1ee13efbc373592c1b22b1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"32B","LoRA Training":"true"}. This record does not assert independent reproduction. ### Qwen3-VL-Thinking Overall Score: 30.8. Result ID: 6a1ee13efbc373592c1b22ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4B"}. This record does not assert independent reproduction. ### LLaVA-Critic Overall Score: 27.2. Result ID: 6a1ee13efbc373592c1b22aa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B","LoRA Training":"true"}. This record does not assert independent reproduction. ### Perception-Judge-Qwen3 Overall Score: 27. Result ID: 6a1ee13efbc373592c1b22b3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. ### LLaVA-1.6 Overall Score: 26.5. Result ID: 6a1ee13efbc373592c1b22a9. [Reporting paper]() Recorded evaluation configuration: {"Size":"34B"}. This record does not assert independent reproduction. ### Gemini-1.0-Pro-Vision Overall Score: 20.7. Result ID: 6a1ee13efbc373592c1b22a6. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Prometheus-V Overall Score: 15.7. Result ID: 6a1ee13efbc373592c1b22a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"13B","LoRA Training":"true"}. This record does not assert independent reproduction. ### Qwen2.5-VL-Instruct Overall Score: 10.3. Result ID: 6a1ee13efbc373592c1b22ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.