Visual Reasoning on MMMU Pro Vision
51.9AccuracyGPT4-o
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT4-oSampling Strategy=avg@8, Inference Temperature=12026.01 | 51.9 | |
| Gemini-2.0-FlashSampling Strategy=avg@8, Inference Temperature=12026.01 | 51.7 | |
| CPPOBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 39 | |
| Vision-SR1Backbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 38.9 | |
| PAPOBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 38.7 | |
| NoisyRolloutBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 38.5 | |
| PerceptionR1Backbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 38.1 | |
| GRPOBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 37.9 | |
| OpenVLThinkerBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 35.5 | |
| Vision-MattersBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 35.5 | |
| Look-BackBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 34.5 | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 33.7 | |
| CPPOBackbone=Qwen2.5-VL-3B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 28.5 | |
| Visionary-R1Backbone=Qwen2.5-VL-3B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 27.9 | |
| PAPOBackbone=Qwen2.5-VL-3B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 26.8 | |
| GRPOBackbone=Qwen2.5-VL-3B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 25.8 | |
| OpenVLThinkerBackbone=Qwen2.5-VL-3B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 25 | |
| Qwen2.5-VL-3BBackbone=Qwen2.5-VL-3B, Sampling Strategy=avg@8, Inference Temperature=12026.01 | 19.9 |