# Mathematical Reasoning on Humanity's Last Exam Math (test) Source: [Mathematical Reasoning on Humanity's Last Exam Math (test)]() Collection index: [llms.txt]() Data updated: 2026-06-18T08:13:29.760Z Evaluation of model performance on the test split of the Humanity's Last Exam Math dataset, focusing on mathematical reasoning accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical reasoning | | Dataset (including any stated split) | Humanity's Last Exam Math (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Skill-MAS-optimized | 29.76 | Not recorded | 6a337b81206542251d7c4fbc | | EvoAgent | 27.38 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-optimized | 26.79 | Not recorded | 6a337b81206542251d7c4fbc | | AFlow | 25 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-init | 24.4 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-optimized | 21.43 | Not recorded | 6a337b81206542251d7c4fbc | | MAS-Orchestra | 21.43 | Not recorded | 6a337b81206542251d7c4fbc | | AFlow | 21.43 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-init | 20.83 | Not recorded | 6a337b81206542251d7c4fbc | | MAS-Orchestra | 20.24 | Not recorded | 6a337b81206542251d7c4fbc | | AOrchestra | 19.64 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-optimized | 18.45 | Not recorded | 6a337b81206542251d7c4fbc | | MAS-Orchestra | 17.26 | Not recorded | 6a337b81206542251d7c4fbc | | MAS2 | 16.56 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-init | 14.88 | Not recorded | 6a337b81206542251d7c4fbc | | MAS-Orchestra | 14.88 | Not recorded | 6a337b81206542251d7c4fbc | | EvoAgent | 14.29 | Not recorded | 6a337b81206542251d7c4fbc | | Skill-MAS-init | 12.5 | Not recorded | 6a337b81206542251d7c4fbc | | AOrchestra | 12.5 | Not recorded | 6a337b81206542251d7c4fbc | | AFlow | 11.31 | Not recorded | 6a337b81206542251d7c4fbc | | MAS2 | 10.42 | Not recorded | 6a337b81206542251d7c4fbc | | EvoAgent | 9.52 | Not recorded | 6a337b81206542251d7c4fbc | | EvoAgent | 8.93 | Not recorded | 6a337b81206542251d7c4fbc | | AOrchestra | 8.93 | Not recorded | 6a337b81206542251d7c4fbc | | AOrchestra | 7.14 | Not recorded | 6a337b81206542251d7c4fbc | | MAS2 | 6.67 | Not recorded | 6a337b81206542251d7c4fbc | | AFlow | 5.95 | Not recorded | 6a337b81206542251d7c4fbc | | MAS2 | 2.33 | Not recorded | 6a337b81206542251d7c4fbc | ### Skill-MAS-optimized Accuracy: 29.76. Result ID: 6a33a89dfbc373592c30d850. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### EvoAgent Accuracy: 27.38. Result ID: 6a33a89dfbc373592c30d84a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### Skill-MAS-optimized Accuracy: 26.79. Result ID: 6a33a89dfbc373592c30d857. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### AFlow Accuracy: 25. Result ID: 6a33a89dfbc373592c30d84c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### Skill-MAS-init Accuracy: 24.4. Result ID: 6a33a89dfbc373592c30d84f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### Skill-MAS-optimized Accuracy: 21.43. Result ID: 6a33a89dfbc373592c30d842. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. ### MAS-Orchestra Accuracy: 21.43. Result ID: 6a33a89dfbc373592c30d84e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### AFlow Accuracy: 21.43. Result ID: 6a33a89dfbc373592c30d853. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### Skill-MAS-init Accuracy: 20.83. Result ID: 6a33a89dfbc373592c30d856. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### MAS-Orchestra Accuracy: 20.24. Result ID: 6a33a89dfbc373592c30d855. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### AOrchestra Accuracy: 19.64. Result ID: 6a33a89dfbc373592c30d852. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### Skill-MAS-optimized Accuracy: 18.45. Result ID: 6a33a89dfbc373592c30d849. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### MAS-Orchestra Accuracy: 17.26. Result ID: 6a33a89dfbc373592c30d840. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. ### MAS2 Accuracy: 16.56. Result ID: 6a33a89dfbc373592c30d84d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### Skill-MAS-init Accuracy: 14.88. Result ID: 6a33a89dfbc373592c30d841. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. ### MAS-Orchestra Accuracy: 14.88. Result ID: 6a33a89dfbc373592c30d847. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### EvoAgent Accuracy: 14.29. Result ID: 6a33a89dfbc373592c30d843. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### Skill-MAS-init Accuracy: 12.5. Result ID: 6a33a89dfbc373592c30d848. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### AOrchestra Accuracy: 12.5. Result ID: 6a33a89dfbc373592c30d84b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Qwen3.5-Plus"}. This record does not assert independent reproduction. ### AFlow Accuracy: 11.31. Result ID: 6a33a89dfbc373592c30d845. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### MAS2 Accuracy: 10.42. Result ID: 6a33a89dfbc373592c30d846. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### EvoAgent Accuracy: 9.52. Result ID: 6a33a89dfbc373592c30d851. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### EvoAgent Accuracy: 8.93. Result ID: 6a33a89dfbc373592c30d83c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. ### AOrchestra Accuracy: 8.93. Result ID: 6a33a89dfbc373592c30d844. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"GPT-5.4-Nano"}. This record does not assert independent reproduction. ### AOrchestra Accuracy: 7.14. Result ID: 6a33a89dfbc373592c30d83d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. ### MAS2 Accuracy: 6.67. Result ID: 6a33a89dfbc373592c30d854. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### AFlow Accuracy: 5.95. Result ID: 6a33a89dfbc373592c30d83e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. ### MAS2 Accuracy: 2.33. Result ID: 6a33a89dfbc373592c30d83f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Meta-agent":"Gemini-3.1-Flash"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.