Multi-Discipline Reasoning on MMMU-Pro (pass@1, avg@8)
51.8Pass@1ADHint
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ADHintBase Model=Qwen3-VL-8B-Instruct2025.12 | 51.8 | 57.9 | |
| GRPOBase Model=Qwen3-VL-8B-Instruct2025.12 | 46.5 | 57.9 | |
| ADHintBase MLLM=Qwen2.5-VL-7B2025.12 | 41.3 | 41.8 | |
| GRPOBase MLLM=Qwen2.5-VL-7B2025.12 | 40.4 | 40 | |
| HintGRPOBase Model=Qwen3-VL-8B-Instruct2025.12 | 38.4 | 46 | |
| GHPOBase MLLM=Qwen2.5-VL-7B2025.12 | 35.3 | 32.3 | |
| ADHintBase MLLM=Qwen2.5-VL-3B2025.12 | 33.3 | 32 | |
| GRPOBase MLLM=Qwen2.5-VL-3B2025.12 | 30.7 | 30.1 | |
| LUFFYBase MLLM=Qwen2.5-VL-7B2025.12 | 30.6 | 29.7 | |
| HintGRPOBase MLLM=Qwen2.5-VL-7B2025.12 | 30.2 | 28.7 | |
| HintGRPOBase MLLM=Qwen2.5-VL-3B2025.12 | 27.5 | 25.1 | |
| GHPOBase MLLM=Qwen2.5-VL-3B2025.12 | 27 | 26.3 | |
| SFT -> RLBase MLLM=Qwen2.5-VL-7B2025.12 | 25.8 | 38.3 | |
| SFTBase MLLM=Qwen2.5-VL-7B2025.12 | 20.6 | 35.6 | |
| LUFFYBase MLLM=Qwen2.5-VL-3B2025.12 | 20.3 | 23.1 | |
| StepHintBase MLLM=Qwen2.5-VL-3B2025.12 | 19.8 | 26 | |
| SFT -> RLBase MLLM=Qwen2.5-VL-3B2025.12 | 18.7 | 30.9 | |
| SFTBase MLLM=Qwen2.5-VL-3B2025.12 | 17.1 | 29.4 | |
| StepHintBase MLLM=Qwen2.5-VL-7B2025.12 | 9.5 | 19.3 |