# Scientific Reasoning on SciBench (pass@3)

Source: [Scientific Reasoning on SciBench (pass@3)](<https://www.sota2.com/research/sota/scientific-reasoning-on-scibench-pass-at-3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T07:48:25.672Z

Evaluation of model performance on scientific reasoning tasks using the SciBench dataset, specifically measured using the pass@3 metric.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Scientific Reasoning |
| Dataset (including any stated split) | SciBench |
| Metric | pass@3 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [pass@3](<https://www.sota2.com/research/sota/scientific-reasoning-on-scibench-pass-at-3.md?metric=pass%403>): Higher is better; unit: PERCENT.

## Recorded results

| Method | pass@3 | Result date | Report paper |
| --- | --- | --- | --- |
| HDPO | 44.36 | Not recorded | 6a210686c10484bac23f010b |
| HDPO | 43.16 | Not recorded | 6a210686c10484bac23f010b |
| Critique-GRPO | 41.38 | Not recorded | 6a210686c10484bac23f010b |
| HDPO | 41.02 | Not recorded | 6a210686c10484bac23f010b |
| GRPO | 36.1 | Not recorded | 6a210686c10484bac23f010b |
| GRPO | 35.54 | Not recorded | 6a210686c10484bac23f010b |
| GRPO | 33.92 | Not recorded | 6a210686c10484bac23f010b |
| LUFFY | 33.37 | Not recorded | 6a210686c10484bac23f010b |
| Oat | 26.31 | Not recorded | 6a210686c10484bac23f010b |
| EDGE-GRPO | 25.79 | Not recorded | 6a210686c10484bac23f010b |
| SimpleRL | 25.72 | Not recorded | 6a210686c10484bac23f010b |
| Base Model | 21.93 | Not recorded | 6a210686c10484bac23f010b |
| Base Model | 19.03 | Not recorded | 6a210686c10484bac23f010b |
| Base Model | 17.7 | Not recorded | 6a210686c10484bac23f010b |

### HDPO

pass@3: 44.36. Result ID: 6a212cecfbc373592c1c6e2f.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### HDPO

pass@3: 43.16. Result ID: 6a212cecfbc373592c1c6e2c.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Number of samples":"16"}. This record does not assert independent reproduction.

### Critique-GRPO

pass@3: 41.38. Result ID: 6a212cecfbc373592c1c6e29.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ac21e209131cea850cf.md>)

Recorded evaluation configuration: {"Backbone":"8B","Number of samples":"16"}. This record does not assert independent reproduction.

### HDPO

pass@3: 41.02. Result ID: 6a212cecfbc373592c1c6e32.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"DS-R1-Llama-8B","Number of samples":"16"}. This record does not assert independent reproduction.

### GRPO

pass@3: 36.1. Result ID: 6a212cecfbc373592c1c6e31.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"DS-R1-Llama-8B","Number of samples":"16"}. This record does not assert independent reproduction.

### GRPO

pass@3: 35.54. Result ID: 6a212cecfbc373592c1c6e2e.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### GRPO

pass@3: 33.92. Result ID: 6a212cecfbc373592c1c6e2b.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Number of samples":"16"}. This record does not assert independent reproduction.

### LUFFY

pass@3: 33.37. Result ID: 6a212cecfbc373592c1c6e28.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79961e209131cea7e682.md>)

Recorded evaluation configuration: {"Backbone":"Qwen-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### Oat

pass@3: 26.31. Result ID: 6a212cecfbc373592c1c6e26.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7be51e209131cea8b201.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### EDGE-GRPO

pass@3: 25.79. Result ID: 6a212cecfbc373592c1c6e25.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a431e209131cea82028.md>)

Recorded evaluation configuration: {"Backbone":"Qwen-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### SimpleRL

pass@3: 25.72. Result ID: 6a212cecfbc373592c1c6e27.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a721e209131cea8345b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Number of samples":"16"}. This record does not assert independent reproduction.

### Base Model

pass@3: 21.93. Result ID: 6a212cecfbc373592c1c6e2a.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Number of samples":"16"}. This record does not assert independent reproduction.

### Base Model

pass@3: 19.03. Result ID: 6a212cecfbc373592c1c6e30.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"DS-R1-Llama-8B","Number of samples":"16"}. This record does not assert independent reproduction.

### Base Model

pass@3: 17.7. Result ID: 6a212cecfbc373592c1c6e2d.

[Reporting paper](<https://www.sota2.com/research/paper/6a210686c10484bac23f010b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B","Number of samples":"16"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
