Vision-Language Compositionality on VisMin Hard (test)
84.47Text Score (T)Qwen2.5-VL-7B + FOCUS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Qwen2.5-VL-7B + FOCUSModel Family=Qwen2.5-VL, Model Size=7B, Evaluation State=Frozen2025.08 | 84.47 | 61.44 | 56.99 | |
| Qwen2.5-VL-7BModel Family=Qwen2.5-VL, Model Size=7B, Evaluation State=Frozen2025.08 | 84.37 | 53.26 | 49.97 | |
| Qwen3-VL-4B + FOCUSModel Family=Qwen3-VL, Model Size=4B, Evaluation State=Frozen2025.08 | 84.11 | 58.31 | 54.32 | |
| Qwen3-VL-4BModel Family=Qwen3-VL, Model Size=4B, Evaluation State=Frozen2025.08 | 83.62 | 49.62 | 46.78 | |
| InternVL3-8BModel Family=InternVL3, Model Size=8B, Evaluation State=Frozen2025.08 | 82.86 | 47.66 | 44.8 | |
| Qwen3-VL-8BModel Family=Qwen3-VL, Model Size=8B, Evaluation State=Frozen2025.08 | 82.86 | 56.65 | 52.95 | |
| Qwen3-VL-8B + FOCUSModel Family=Qwen3-VL, Model Size=8B, Evaluation State=Frozen2025.08 | 82.77 | 59.13 | 55.01 | |
| InternVL3-8B + FOCUSModel Family=InternVL3, Model Size=8B, Evaluation State=Frozen2025.08 | 82.54 | 52.66 | 48.89 | |
| LLaVA-OneVision-7BModel Family=LLaVA-OV, Model Size=7B, Evaluation State=Frozen2025.08 | 80.61 | 44.42 | 38.4 | |
| LLaVA-OneVision-7B + FOCUSModel Family=LLaVA-OV, Model Size=7B, Evaluation State=Frozen2025.08 | 80.44 | 57.89 | 49.4 | |
| Qwen2.5-VL-3B + FOCUSModel Family=Qwen2.5-VL, Model Size=3B, Evaluation State=Frozen2025.08 | 78.2 | 28.86 | 25.29 | |
| Qwen2.5-VL-3BModel Family=Qwen2.5-VL, Model Size=3B, Evaluation State=Frozen2025.08 | 77.38 | 27.09 | 22.93 | |
| InternVL3-2BModel Family=InternVL3, Model Size=2B, Evaluation State=Frozen2025.08 | 65.53 | 6.84 | 6.17 | |
| InternVL3-2B + FOCUSModel Family=InternVL3, Model Size=2B, Evaluation State=Frozen2025.08 | 64.87 | 26.77 | 23.96 | |
| LLaVA-OneVision-0.5B + FOCUSModel Family=LLaVA-OV, Model Size=0.5B, Evaluation State=Frozen2025.08 | 2.2 | 18.41 | 0.59 | |
| LLaVA-OneVision-0.5BModel Family=LLaVA-OV, Model Size=0.5B, Evaluation State=Frozen2025.08 | 1.7 | 20.12 | 0.51 |