Accuracy on MMMU
83.4Accuracy (MMMU)S1-VL-32B-RL
Evaluation Results
| Method | Links | |
|---|---|---|
| S1-VL-32B-RL2026.06 | 83.4 | |
| S1-Omni-Image2026.06 | 82.67 | |
| S1-VL-32B-SFT2026.06 | 82.5 | |
| GPT-52026.06 | 81.22 | |
| Qwen3-VL-235B-A22B-Thinking2026.06 | 77.89 | |
| Qwen3-VL-32B-Thinking2026.06 | 76 | |
| BF16Precision=BF162026.06 | 60.11 | |
| Qwen3-VL-32B-InstructBackbone model=Qwen3-VL-32B-Instruct2026.05 | 58 | |
| MODEQuantization=W32026.06 | 57.33 | |
| IVR-R1Backbone model=Qwen3-VL-4B2026.05 | 56 | |
| VEQ-MAQuantization=W32026.06 | 55.49 | |
| MC-MoEQuantization=W32026.06 | 54.56 | |
| Vision-R1Backbone model=Qwen3-VL-4B2026.05 | 54.1 | |
| IVR-R1Backbone model=Qwen2.5-VL-7B2026.05 | 53.6 | |
| GPTQQuantization=W32026.06 | 52.56 | |
| Vision-SR1Backbone model=Qwen3-VL-4B2026.05 | 52.4 | |
| QWEN2.5-VL-7B-INSTRUCT (PALETTE)Allowed Domain=PH2026.05 | 52 | |
| QWEN2.5-VL-7B-INSTRUCT (PALETTE)Allowed Domain=PV2026.05 | 51.7 | |
| QWEN2.5-VL-7B-INSTRUCT (PALETTE)Allowed Domain=HS2026.05 | 51.3 | |
| QWEN2.5-VL-7B-INSTRUCT (PALETTE)Allowed Domain=Fraud2026.05 | 51.1 | |
| Vision-R1Backbone model=Qwen2.5-VL-7B2026.05 | 50.9 | |
| QWEN2.5-VL-7B-INSTRUCTAllowed Domain=Base2026.05 | 50.9 | |
| QWEN2.5-VL-7B-INSTRUCT (PALETTE)Allowed Domain=Sex2026.05 | 50.9 | |
| QWEN2.5-VL-7B-INSTRUCT (PALETTE)Allowed Domain=IA2026.05 | 50.8 | |
| SplitQBackbone=Qwen2.5-VL-7B, Bits=W4A82026.05 | 49.1 | |
| MODEQuantization=W22026.06 | 48.67 | |
| Thyme-VL2026.06 | 48.66 | |
| Supervised Fine-tuning (before RL)Backbone model=Qwen3-VL-4B2026.05 | 48.4 | |
| Vision-SR1Backbone model=Qwen2.5-VL-7B2026.05 | 47.6 | |
| Zero-shot Inference (before RL)Backbone model=Qwen3-VL-4B2026.05 | 47.4 | |
| SplitQBackbone=Qwen2.5-VL-7B, Bits=W4A42026.05 | 46.9 | |
| MASQBackbone=Qwen2.5-VL-3B, Bits=W4A82026.05 | 46.7 | |
| FP16Backbone=Qwen2.5-VL-7B, Bits=W16A162026.05 | 46.7 | |
| SplitQBackbone=Qwen2.5-VL-3B, Bits=W4A82026.05 | 46.3 | |
| VEQ-MAQuantization=W22026.06 | 46.11 | |
| Supervised Fine-tuning (before RL)Backbone model=Qwen2.5-VL-7B2026.05 | 46.1 | |
| MC-MoEQuantization=W22026.06 | 44.89 | |
| SplitQBackbone=Qwen2.5-VL-7B, Bits=W3A32026.05 | 43.9 | |
| SplitQBackbone=Qwen2.5-VL-3B, Bits=W4A42026.05 | 43.7 | |
| MASQBackbone=Qwen2.5-VL-7B, Bits=W4A82026.05 | 43.4 | |
| MBQBackbone=Qwen2.5-VL-7B, Bits=W4A82026.05 | 43.3 | |
| FP16Backbone=Qwen2.5-VL-3B, Bits=W16A162026.05 | 42.2 | |
| MBQBackbone=Qwen2.5-VL-3B, Bits=W4A82026.05 | 41.2 | |
| SplitQBackbone=Qwen2.5-VL-3B, Bits=W3A32026.05 | 39.8 | |
| LLaVA-NEXT-3BBackbone=LLaVA-NEXT-3B2026.05 | 39.6 | |
| LBRBackbone=LLaVA-NEXT-3B2026.05 | 38.8 | |
| LLaVA-1.5 + FG-CLIP 2Vision Encoder=FG-CLIP 2, LMM Base Architecture=LLaVA-1.52025.10 | 38.1 | |
| LBRBackbone=LLaVA-1.5-13B2026.05 | 37.9 | |
| SQBackbone=Qwen2.5-VL-7B, Bits=W4A82026.05 | 37.8 | |
| LBRBackbone=LLaVA-1.5-7B2026.05 | 37.2 | |
| LLaVA-1.5 + SigLIP 2Vision Encoder=SigLIP 2, LMM Base Architecture=LLaVA-1.52025.10 | 37 | |
| LLaVA-1.5 + Meta CLIP 2Vision Encoder=Meta CLIP 2, LMM Base Architecture=LLaVA-1.52025.10 | 37 | |
| LLaVA-1.5-13BBackbone=LLaVA-1.5-13B2026.05 | 36.7 | |
| LLaVA-1.5-7BBackbone=LLaVA-1.5-7B2026.05 | 35.7 | |
| LLaVA-1.5 + CLIPVision Encoder=CLIP, LMM Base Architecture=LLaVA-1.52025.10 | 35.7 | |
| SplitQBackbone=Qwen2.5-VL-7B, Bits=W3A22026.05 | 35.6 | |
| Qwen2.5-VL-72B-InstructBackbone model=Qwen2.5-VL-72B-Instruct2026.05 | 34.9 | |
| SplitQBackbone=Qwen2.5-VL-3B, Bits=W3A22026.05 | 33.1 | |
| OPPOTraining=Oriented Pickup Preference Optimization, Base Model=LLaVA-NeXT2026.06 | 32.3 | |
| LLaVA-NeXT (Baseline)Training=Vanilla2026.06 | 31.8 | |
| mDPOTraining=Preference Optimization, Base Model=LLaVA-NeXT2026.06 | 31.5 | |
| CHiPTraining=Preference Optimization, Base Model=LLaVA-NeXT2026.06 | 31 | |
| DPOTraining=Preference Optimization, Base Model=LLaVA-NeXT2026.06 | 30.9 | |
| MASQBackbone=Qwen2.5-VL-3B, Bits=W4A42026.05 | 26.7 | |
| MBQBackbone=Qwen2.5-VL-7B, Bits=W4A42026.05 | 26.7 | |
| SQBackbone=Qwen2.5-VL-3B, Bits=W4A82026.05 | 25.6 | |
| MBQBackbone=Qwen2.5-VL-3B, Bits=W4A42026.05 | 25 | |
| MASQBackbone=Qwen2.5-VL-7B, Bits=W4A42026.05 | 25 | |
| SQBackbone=Qwen2.5-VL-7B, Bits=W4A42026.05 | 24.8 | |
| SQBackbone=Qwen2.5-VL-3B, Bits=W4A42026.05 | 23.3 | |
| Qwen2.5-VL-32B-InstructBackbone model=Qwen2.5-VL-32B-Instruct2026.05 | 22.9 | |
| GPTQQuantization=W22026.06 | 22.78 | |
| Zero-shot Inference (before RL)Backbone model=Qwen2.5-VL-7B2026.05 | 21.6 |