Task#1 on STAR
99.88AccuracyQwen3-VL-8B S2(KGRPO)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-VL-8B S2(KGRPO)Backbone=Qwen3-VL-8B, Setting=S2(KGRPO)2025.10 | 99.88 | 96.4 | |
| Qwen3-VL-4B S2(KGRPO)Backbone=Qwen3-VL-4B, Setting=S2(KGRPO)2025.10 | 99.75 | 95.36 | |
| Qwen3-VL-2B S2(KGRPO)Backbone=Qwen3-VL-2B, Setting=S2(KGRPO)2025.10 | 99.38 | 94.49 | |
| LLaVA-NEXT-34B S2(SimPO)Backbone=LLaVA-NEXT-34B, Setting=S2(SimPO)2025.10 | 98.25 | 92.96 | |
| GPT-4oBackbone=GPT-4o2025.10 | 94.25 | — | |
| KGRPOBackbone=Qwen2.5-VL-7B, Settings=S2(KGRPO)2025.10 | 79.88 | 86.4 | |
| Qwen2.5-VL-7B S2(KGRPO)Backbone=Qwen2.5-VL-7B, Setting=S2(KGRPO)2025.10 | 79.88 | 86.4 | |
| S1(Single)Backbone=Qwen2.5-VL-7B, Settings=S1(Single)2025.10 | 77.13 | 82.47 | |
| Qwen2.5-VL-32B S2(SimPO)Backbone=Qwen2.5-VL-32B, Setting=S2(SimPO)2025.10 | 76.37 | 82.01 | |
| GRPOBackbone=Qwen2.5-VL-7B, Settings=S2(GRPO)2025.10 | 75.25 | 83.07 | |
| KGRPOBackbone=Qwen2.5-VL-3B, Settings=S2(KGRPO)2025.10 | 75 | 81.49 | |
| Qwen2.5-VL-3B S2(KGRPO)Backbone=Qwen2.5-VL-3B, Setting=S2(KGRPO)2025.10 | 75 | 81.49 | |
| SimPOBackbone=Qwen2.5-VL-3B, Settings=S2(SimPO)2025.10 | 71 | 79.03 | |
| SimPOBackbone=Qwen2.5-VL-7B, Settings=S2(SimPO)2025.10 | 69.63 | 82.96 | |
| GPT-4o-miniBackbone=GPT-4o-mini, Settings=Zero-shot2025.10 | 67.5 | — | |
| GPT-4o-miniBackbone=GPT-4o-mini2025.10 | 67.5 | — | |
| LLaVA-1.5/NEXT-7B S2(SimPO)Backbone=LLaVA-1.5/NEXT-7B, Setting=S2(SimPO)2025.10 | 67.38 | 48.86 | |
| DPOBackbone=Qwen2.5-VL-7B, Settings=S2(DPO)2025.10 | 66.5 | 82.11 | |
| ORPOBackbone=Qwen2.5-VL-7B, Settings=S2(ORPO)2025.10 | 65.75 | 81.96 | |
| S1(Full)Backbone=Qwen2.5-VL-7B, Settings=S1(Full)2025.10 | 64.88 | 81.79 | |
| GRPOBackbone=Qwen2.5-VL-3B, Settings=S2(GRPO)2025.10 | 60 | 76.49 | |
| DPOBackbone=Qwen2.5-VL-3B, Settings=S2(DPO)2025.10 | 55.5 | 73.28 | |
| S1(Single)Backbone=Qwen2.5-VL-3B, Settings=S1(Single)2025.10 | 51 | 62.07 | |
| GPT-4oBackbone=GPT-4o, Settings=Zero-shot2025.10 | 43.75 | — | |
| Qwen3-VL-30BBackbone=Qwen3-VL-30B2025.10 | 43.75 | — | |
| S1(Full)Backbone=Qwen2.5-VL-3B, Settings=S1(Full)2025.10 | 42.75 | 52.67 | |
| ORPOBackbone=Qwen2.5-VL-3B, Settings=S2(ORPO)2025.10 | 39 | 64.79 | |
| Qwen2.5VL-72BBackbone=Qwen2.5VL-72B2025.10 | 38.25 | — | |
| GPT-4vBackbone=GPT-4v, Settings=Zero-shot2025.10 | 37.75 | — | |
| GPT-4vBackbone=GPT-4v2025.10 | 37.75 | — | |
| QVQ-72BBackbone=QVQ-72B2025.10 | 30.75 | — | |
| Zero-shotBackbone=Qwen2.5-VL-3B, Settings=Zero-shot2025.10 | 18.25 | — | |
| Zero-shotBackbone=Qwen2.5-VL-7B, Settings=Zero-shot2025.10 | 6.13 | — |