STEM & Puzzle on MMMU (val)
79.7AccuracyM2Note
Evaluation results
| Method | Links | |
|---|---|---|
| M2NoteBackbone=Qwen3-VL-Plus, Evolving Mode=Self-Evolving, Cost=≈ $7, Mem=28, Len=4622026.07 | 79.7 | |
| GPT5 miniModel Source Category=Closed-source VLMs, Evaluation Mode=Thinking Mode2026.01 | 79 | |
| Qwen3-VL 32BModel Source Category=Open-weight VLMs, Model Parameters=32B, Evaluation Mode=Thinking Mode2026.01 | 78.1 | |
| VanillaBackbone=Qwen3-VL-Plus, Evolving Mode=Self-Evolving2026.07 | 77.9 | |
| M2NoteBackbone=GPT-5.4, Evolving Mode=Self-Evolving, Cost=≈ $7, Mem=28, Len=4622026.07 | 77.9 | |
| Gemini-2.5 FlashModel Source Category=Closed-source VLMs, Evaluation Mode=Thinking Mode2026.01 | 77.7 | |
| VanillaBackbone=GPT-5.4, Evolving Mode=Self-Evolving2026.07 | 77.1 | |
| Qwen3-VL 30B-A3BModel Source Category=Open-weight VLMs, Model Parameters=30B-A3B, Evaluation Mode=Thinking Mode2026.01 | 76 | |
| Qwen3-VL 8BModel Source Category=Open-weight VLMs, Model Parameters=8B, Evaluation Mode=Thinking Mode2026.01 | 74.1 | |
| MMFineReason-8BModel Source Category=Ours, Model Parameters=8B, Evaluation Mode=Thinking Mode2026.01 | 71.3 | |
| MMFineReason-4BModel Source Category=Ours, Model Parameters=4B, Evaluation Mode=Thinking Mode2026.01 | 69.6 | |
| CoT+M2NoteBackbone=Qwen3-VL-8B-Instruct, Evolving Mode=Self-Evolving, Cost=≈ $4, Mem=32, Len=3152026.07 | 69.4 | |
| DPEBackbone=Qwen3-VL-8B-Instruct, Evolving Mode=Self-Evolving, Cost=> $5002026.07 | 69.1 | |
| M2NoteBackbone (Tuning)=Qwen3-VL-8B-Instruct, Backbone (Tuner)=Qwen3-VL-Plus, Evolving Mode=Cross-Model Evolving, Cost=≈ $5, Mem=43, Len=4452026.07 | 68.9 | |
| CoTBackbone=Qwen3-VL-8B-Instruct, Evolving Mode=Self-Evolving2026.07 | 68.8 | |
| M2NoteBackbone=Qwen3-VL-8B-Instruct, Evolving Mode=Self-Evolving, Cost=≈ $3, Mem=50, Len=2612026.07 | 68.4 | |
| VanillaBackbone=Qwen3-VL-8B-Instruct, Evolving Mode=Self-Evolving2026.07 | 67.1 | |
| VanillaBackbone (Tuning)=Qwen3-VL-8B-Instruct, Backbone (Tuner)=Qwen3-VL-Plus, Evolving Mode=Cross-Model Evolving2026.07 | 67.1 | |
| HoneyBee 8BModel Source Category=Open-source VLMs, Model Parameters=8B, Evaluation Mode=Thinking Mode2026.01 | 63.1 | |
| MMR1 8BModel Source Category=Open-source VLMs, Model Parameters=8B, Evaluation Mode=Thinking Mode2026.01 | 62.8 | |
| Vision-ZeroEvolving Mode=Self-Evolving, Cost=Basically2026.07 | 58.8 | |
| OMR 7BModel Source Category=Open-source VLMs, Model Parameters=7B, Evaluation Mode=Thinking Mode2026.01 | 57.8 | |
| VisPlayEvolving Mode=Self-Evolving2026.07 | 54.9 | |
| MMFineReason-2BModel Source Category=Ours, Model Parameters=2B, Evaluation Mode=Thinking Mode2026.01 | 54.8 | |
| iReasonerEvolving Mode=Self-Evolving2026.07 | 52.4 | |
| EvoLMMEvolving Mode=Self-Evolving, Cost=> $5002026.07 | 52 |