# Graduate-level Reasoning on SuperGPQA (Pass@1) Source: [Graduate-level Reasoning on SuperGPQA (Pass@1)]() Collection index: [llms.txt]() Data updated: 2026-03-12T08:55:26.214Z Evaluation of model performance on the Graduate-level Reasoning task using the SuperGPQA dataset, measured by Pass@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Graduate-level Reasoning | | Dataset (including any stated split) | SuperGPQA | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | Rule+CER | 36 | Not recorded | 69b248afd030084c22e2fa78 | | Rule-based | 35.1 | Not recorded | 69b248afd030084c22e2fa78 | | CER | 35 | Not recorded | 69b248afd030084c22e2fa78 | | General-verifier | 34.9 | Not recorded | 69b248afd030084c22e2fa78 | | Exact-match | 33.5 | Not recorded | 69b248afd030084c22e2fa78 | | VeriFree | 32.4 | Not recorded | 69b248afd030084c22e2fa78 | | Rule-based | 32.2 | Not recorded | 69b248afd030084c22e2fa78 | | CER | 32.1 | Not recorded | 69b248afd030084c22e2fa78 | | Rule+CER | 31.3 | Not recorded | 69b248afd030084c22e2fa78 | | VeriFree | 30.9 | Not recorded | 69b248afd030084c22e2fa78 | | General-verifier | 30.9 | Not recorded | 69b248afd030084c22e2fa78 | | Base | 27 | Not recorded | 69b248afd030084c22e2fa78 | | Exact-match | 24.2 | Not recorded | 69b248afd030084c22e2fa78 | | Base | 21 | Not recorded | 69b248afd030084c22e2fa78 | ### Rule+CER Pass@1: 36. Result ID: 69b27f656dbb7075b95d3f92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Rule-based Pass@1: 35.1. Result ID: 69b27f656dbb7075b95d3f8e. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### CER Pass@1: 35. Result ID: 69b27f656dbb7075b95d3f91. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### General-verifier Pass@1: 34.9. Result ID: 69b27f656dbb7075b95d3f90. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Exact-match Pass@1: 33.5. Result ID: 69b27f656dbb7075b95d3f8d. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### VeriFree Pass@1: 32.4. Result ID: 69b27f656dbb7075b95d3f8f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Rule-based Pass@1: 32.2. Result ID: 69b27f656dbb7075b95d3f87. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### CER Pass@1: 32.1. Result ID: 69b27f656dbb7075b95d3f8a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Rule+CER Pass@1: 31.3. Result ID: 69b27f656dbb7075b95d3f8b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### VeriFree Pass@1: 30.9. Result ID: 69b27f656dbb7075b95d3f88. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### General-verifier Pass@1: 30.9. Result ID: 69b27f656dbb7075b95d3f89. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Base Pass@1: 27. Result ID: 69b27f656dbb7075b95d3f8c. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Exact-match Pass@1: 24.2. Result ID: 69b27f656dbb7075b95d3f86. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. ### Base Pass@1: 21. Result ID: 69b27f656dbb7075b95d3f85. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.