# STEM & Puzzle on MMMU (val) Source: [STEM & Puzzle on MMMU (val)]() Collection index: [llms.txt]() Data updated: 2026-07-02T08:29:22.655Z Evaluation of model performance on the STEM & Puzzle task using the validation split of the MMMU dataset, measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | STEM & Puzzle | | Dataset (including any stated split) | MMMU (val) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | M2Note | 79.7 | Not recorded | 6a45f070dc71372d7795a090 | | GPT5 mini | 79 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | Qwen3-VL 32B | 78.1 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | Vanilla | 77.9 | Not recorded | 6a45f070dc71372d7795a090 | | M2Note | 77.9 | Not recorded | 6a45f070dc71372d7795a090 | | Gemini-2.5 Flash | 77.7 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | Vanilla | 77.1 | Not recorded | 6a45f070dc71372d7795a090 | | Qwen3-VL 30B-A3B | 76 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | Qwen3-VL 8B | 74.1 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | MMFineReason-8B | 71.3 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | MMFineReason-4B | 69.6 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | CoT+M2Note | 69.4 | Not recorded | 6a45f070dc71372d7795a090 | | DPE | 69.1 | Not recorded | 6a45f070dc71372d7795a090 | | M2Note | 68.9 | Not recorded | 6a45f070dc71372d7795a090 | | CoT | 68.8 | Not recorded | 6a45f070dc71372d7795a090 | | M2Note | 68.4 | Not recorded | 6a45f070dc71372d7795a090 | | Vanilla | 67.1 | Not recorded | 6a45f070dc71372d7795a090 | | Vanilla | 67.1 | Not recorded | 6a45f070dc71372d7795a090 | | HoneyBee 8B | 63.1 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | MMR1 8B | 62.8 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | Vision-Zero | 58.8 | Not recorded | 6a45f070dc71372d7795a090 | | OMR 7B | 57.8 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | VisPlay | 54.9 | Not recorded | 6a45f070dc71372d7795a090 | | MMFineReason-2B | 54.8 | Not recorded | 697c3b66a8b01e76cc8da6e1 | | iReasoner | 52.4 | Not recorded | 6a45f070dc71372d7795a090 | | EvoLMM | 52 | Not recorded | 6a45f070dc71372d7795a090 | ### M2Note Accuracy: 79.7. Result ID: 6a46216160b88bb339850dbe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-Plus","Evolving Mode":"Self-Evolving","Cost":"≈ $7","Mem":"28","Len":"462"}. This record does not assert independent reproduction. ### GPT5 mini Accuracy: 79. Result ID: 6997797c5698665ec4cdd04c. [Reporting paper]() Recorded evaluation configuration: {"Model Source Category":"Closed-source VLMs","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### Qwen3-VL 32B Accuracy: 78.1. Result ID: 6997797c5698665ec4cdd04f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Open-weight VLMs","Model Parameters":"32B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 77.9. Result ID: 6a46216160b88bb339850dbd. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-Plus","Evolving Mode":"Self-Evolving"}. This record does not assert independent reproduction. ### M2Note Accuracy: 77.9. Result ID: 6a46216160b88bb339850dc0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.4","Evolving Mode":"Self-Evolving","Cost":"≈ $7","Mem":"28","Len":"462"}. This record does not assert independent reproduction. ### Gemini-2.5 Flash Accuracy: 77.7. Result ID: 6997797c5698665ec4cdd04b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Closed-source VLMs","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 77.1. Result ID: 6a46216160b88bb339850dbf. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.4","Evolving Mode":"Self-Evolving"}. This record does not assert independent reproduction. ### Qwen3-VL 30B-A3B Accuracy: 76. Result ID: 6997797c5698665ec4cdd04e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Open-weight VLMs","Model Parameters":"30B-A3B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### Qwen3-VL 8B Accuracy: 74.1. Result ID: 6997797c5698665ec4cdd04d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Open-weight VLMs","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### MMFineReason-8B Accuracy: 71.3. Result ID: 6997797c5698665ec4cdd055. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Ours","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### MMFineReason-4B Accuracy: 69.6. Result ID: 6997797c5698665ec4cdd054. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Ours","Model Parameters":"4B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### CoT+M2Note Accuracy: 69.4. Result ID: 6a46216160b88bb339850dbc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B-Instruct","Evolving Mode":"Self-Evolving","Cost":"≈ $4","Mem":"32","Len":"315"}. This record does not assert independent reproduction. ### DPE Accuracy: 69.1. Result ID: 6a46216160b88bb339850db9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B-Instruct","Evolving Mode":"Self-Evolving","Cost":"\> $500"}. This record does not assert independent reproduction. ### M2Note Accuracy: 68.9. Result ID: 6a46216160b88bb339850dc2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone (Tuning)":"Qwen3-VL-8B-Instruct","Backbone (Tuner)":"Qwen3-VL-Plus","Evolving Mode":"Cross-Model Evolving","Cost":"≈ $5","Mem":"43","Len":"445"}. This record does not assert independent reproduction. ### CoT Accuracy: 68.8. Result ID: 6a46216160b88bb339850dbb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B-Instruct","Evolving Mode":"Self-Evolving"}. This record does not assert independent reproduction. ### M2Note Accuracy: 68.4. Result ID: 6a46216160b88bb339850dba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B-Instruct","Evolving Mode":"Self-Evolving","Cost":"≈ $3","Mem":"50","Len":"261"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 67.1. Result ID: 6a46216160b88bb339850db8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B-Instruct","Evolving Mode":"Self-Evolving"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 67.1. Result ID: 6a46216160b88bb339850dc1. [Reporting paper]() Recorded evaluation configuration: {"Backbone (Tuning)":"Qwen3-VL-8B-Instruct","Backbone (Tuner)":"Qwen3-VL-Plus","Evolving Mode":"Cross-Model Evolving"}. This record does not assert independent reproduction. ### HoneyBee 8B Accuracy: 63.1. Result ID: 6997797c5698665ec4cdd051. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Source Category":"Open-source VLMs","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### MMR1 8B Accuracy: 62.8. Result ID: 6997797c5698665ec4cdd050. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Open-source VLMs","Model Parameters":"8B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### Vision-Zero Accuracy: 58.8. Result ID: 6a46216160b88bb339850db4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evolving Mode":"Self-Evolving","Cost":"Basically"}. This record does not assert independent reproduction. ### OMR 7B Accuracy: 57.8. Result ID: 6997797c5698665ec4cdd052. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Source Category":"Open-source VLMs","Model Parameters":"7B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### VisPlay Accuracy: 54.9. Result ID: 6a46216160b88bb339850db7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evolving Mode":"Self-Evolving"}. This record does not assert independent reproduction. ### MMFineReason-2B Accuracy: 54.8. Result ID: 6997797c5698665ec4cdd053. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Source Category":"Ours","Model Parameters":"2B","Evaluation Mode":"Thinking Mode"}. This record does not assert independent reproduction. ### iReasoner Accuracy: 52.4. Result ID: 6a46216160b88bb339850db6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evolving Mode":"Self-Evolving"}. This record does not assert independent reproduction. ### EvoLMM Accuracy: 52. Result ID: 6a46216160b88bb339850db5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evolving Mode":"Self-Evolving","Cost":"\> $500"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.