# Mathematical Reasoning on MINERVA (Pass@8) Source: [Mathematical Reasoning on MINERVA (Pass@8)]() Collection index: [llms.txt]() Data updated: 2026-05-11T10:31:22.191Z Evaluation of model performance on complex mathematical reasoning problems using the Minerva dataset, specifically measured using sampling rates corresponding to k=8 (Pass@8, Avg@8). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical reasoning | | Dataset (including any stated split) | MINERVA | | Metric | Avg Score @8 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Avg Score @8](): Higher is better; unit: PERCENT. - [Pass@8 Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Avg Score @8 | Result date | Report paper | | --- | --- | --- | --- | | vOPDtop-k | 37.6 | Not recorded | 6a0162c0dbc836b87d8233b0 | | vOPDfull-V | 37.2 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPDfull-V | 36.3 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPD | 35.3 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPDtop-k | 35.3 | Not recorded | 6a0162c0dbc836b87d8233b0 | | vOPDfull-V | 26.2 | Not recorded | 6a0162c0dbc836b87d8233b0 | | vOPDtop-k | 25.2 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPDfull-V | 25 | Not recorded | 6a0162c0dbc836b87d8233b0 | | vOPDtop-k | 23.9 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPDtop-k | 23.4 | Not recorded | 6a0162c0dbc836b87d8233b0 | | vOPDfull-V | 22.5 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPD | 22.2 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPDfull-V | 21.7 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPDtop-k | 21.5 | Not recorded | 6a0162c0dbc836b87d8233b0 | | OPD | 18.4 | Not recorded | 6a0162c0dbc836b87d8233b0 | | Student | 15.9 | Not recorded | 6a0162c0dbc836b87d8233b0 | | Student | 13.8 | Not recorded | 6a0162c0dbc836b87d8233b0 | | Student | 11.6 | Not recorded | 6a0162c0dbc836b87d8233b0 | ### vOPDtop-k Avg Score @8: 37.6. Result ID: 6a01aff747f34ce924e43f5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-4B → Qwen3-4B-Base"}. This record does not assert independent reproduction. ### vOPDfull-V Avg Score @8: 37.2. Result ID: 6a01aff747f34ce924e43f5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-4B → Qwen3-4B-Base"}. This record does not assert independent reproduction. ### OPDfull-V Avg Score @8: 36.3. Result ID: 6a01aff747f34ce924e43f5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-4B → Qwen3-4B-Base"}. This record does not assert independent reproduction. ### OPD Avg Score @8: 35.3. Result ID: 6a01aff747f34ce924e43f59. [Reporting paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-4B → Qwen3-4B-Base"}. This record does not assert independent reproduction. ### OPDtop-k Avg Score @8: 35.3. Result ID: 6a01aff747f34ce924e43f5a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Setup":"Qwen3-4B → Qwen3-4B-Base"}. This record does not assert independent reproduction. ### vOPDfull-V Avg Score @8: 26.2. Result ID: 6a01aff747f34ce924e43f57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-1.7B → Qwen3-1.7B-Base"}. This record does not assert independent reproduction. ### vOPDtop-k Avg Score @8: 25.2. Result ID: 6a01aff747f34ce924e43f56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-1.7B → Qwen3-1.7B-Base"}. This record does not assert independent reproduction. ### OPDfull-V Avg Score @8: 25. Result ID: 6a01aff747f34ce924e43f55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-1.7B → Qwen3-1.7B-Base"}. This record does not assert independent reproduction. ### vOPDtop-k Avg Score @8: 23.9. Result ID: 6a01aff747f34ce924e43f62. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Olmo-3-7B-Think → Olmo-3-7B-Base"}. This record does not assert independent reproduction. ### OPDtop-k Avg Score @8: 23.4. Result ID: 6a01aff747f34ce924e43f54. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Setup":"Qwen3-1.7B → Qwen3-1.7B-Base"}. This record does not assert independent reproduction. ### vOPDfull-V Avg Score @8: 22.5. Result ID: 6a01aff747f34ce924e43f63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Olmo-3-7B-Think → Olmo-3-7B-Base"}. This record does not assert independent reproduction. ### OPD Avg Score @8: 22.2. Result ID: 6a01aff747f34ce924e43f53. [Reporting paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-1.7B → Qwen3-1.7B-Base"}. This record does not assert independent reproduction. ### OPDfull-V Avg Score @8: 21.7. Result ID: 6a01aff747f34ce924e43f61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Olmo-3-7B-Think → Olmo-3-7B-Base"}. This record does not assert independent reproduction. ### OPDtop-k Avg Score @8: 21.5. Result ID: 6a01aff747f34ce924e43f60. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Setup":"Olmo-3-7B-Think → Olmo-3-7B-Base"}. This record does not assert independent reproduction. ### OPD Avg Score @8: 18.4. Result ID: 6a01aff747f34ce924e43f5f. [Reporting paper]() Recorded evaluation configuration: {"Model Setup":"Olmo-3-7B-Think → Olmo-3-7B-Base"}. This record does not assert independent reproduction. ### Student Avg Score @8: 15.9. Result ID: 6a01aff747f34ce924e43f58. [Reporting paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-4B → Qwen3-4B-Base"}. This record does not assert independent reproduction. ### Student Avg Score @8: 13.8. Result ID: 6a01aff747f34ce924e43f52. [Reporting paper]() Recorded evaluation configuration: {"Model Setup":"Qwen3-1.7B → Qwen3-1.7B-Base"}. This record does not assert independent reproduction. ### Student Avg Score @8: 11.6. Result ID: 6a01aff747f34ce924e43f5e. [Reporting paper]() Recorded evaluation configuration: {"Model Setup":"Olmo-3-7B-Think → Olmo-3-7B-Base"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.