Scientific Reasoning on Materials
78Average Score @16 (1h)GRPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GRPOModel Scale=4B2026.04 | 78 | 78.9 | — | 80.1 | |
| SRPOModel Scale=4B2026.04 | 75.7 | 79.1 | — | 81.3 | |
| SRPOModel Scale=8B2026.04 | 74.9 | 79.2 | — | 81.5 | |
| GRPOModel Scale=8B2026.04 | 74.7 | 77.6 | — | 77.8 | |
| GRPOBackbone=Qwen3-8B, Algorithm=GRPO, Policy Type=off-policy2026.01 | 74.3 | 77.1 | — | — | |
| SDPOModel Scale=4B2026.04 | 74.3 | 74.3 | — | 74.3 | |
| GRPO (on-policy)Backbone=Qwen3-8B, Algorithm=GRPO, Policy Type=on-policy2026.01 | 73.9 | 74.1 | — | — | |
| SDPO (on-policy)Backbone=Olmo3-7B-Instruct, Algorithm=SDPO, Policy Type=on-policy2026.01 | 73.7 | 79.1 | — | — | |
| GRPO (on-policy)Backbone=Olmo3-7B-Instruct, Algorithm=GRPO, Policy Type=on-policy2026.01 | 73.3 | 73.5 | — | — | |
| SDPO (on-policy)Backbone=Qwen3-8B, Algorithm=SDPO, Policy Type=on-policy2026.01 | 72.1 | 78.4 | — | — | |
| GRPOBackbone=Olmo3-7B-Instruct, Algorithm=GRPO, Policy Type=off-policy2026.01 | 70.9 | 75 | — | — | |
| SDPOModel Scale=8B2026.04 | 68.1 | 76.6 | — | 76.6 | |
| Qwen3-4BModel Scale=4B2026.04 | 61.2 | — | — | — | |
| Qwen3-8BModel Scale=8B2026.04 | 59.3 | — | — | — | |
| Qwen3-8BBackbone=Qwen3-8B, Algorithm=None, Policy Type=N/A2026.01 | 58.9 | — | — | — | |
| Olmo3-7B-InstructBackbone=Olmo3-7B-Instruct, Algorithm=None, Policy Type=N/A2026.01 | 36.7 | — | — | — | |
| GRPOBackbone=Qwen3-8B, Training Time=1h2026.01 | — | — | 74.3 | — | |
| GRPOBackbone=Qwen3-8B, Training Time=5h2026.01 | — | — | 77.1 | — | |
| GRPOBackbone=Olmo3-7B-Instruct, Training Time=1h2026.01 | — | — | 73.8 | — | |
| GRPOBackbone=Olmo3-7B-Instruct, Training Time=5h2026.01 | — | — | 78.1 | — | |
| GRPO (on-policy)Backbone=Qwen3-8B, Training Time=1h2026.01 | — | — | 73.3 | — | |
| GRPO (on-policy)Backbone=Qwen3-8B, Training Time=5h2026.01 | — | — | 75.8 | — | |
| GRPO (on-policy)Backbone=Olmo3-7B-Instruct, Training Time=1h2026.01 | — | — | 67.9 | — | |
| GRPO (on-policy)Backbone=Olmo3-7B-Instruct, Training Time=5h2026.01 | — | — | 74.4 | — | |
| Olmo3-7B-InstructTraining Time=0h2026.01 | — | — | 36.7 | — | |
| Qwen3-8BTraining Time=0h2026.01 | — | — | 58.9 | — | |
| SDPO (on-policy)Backbone=Qwen3-8B, Training Time=1h2026.01 | — | — | 72.1 | — | |
| SDPO (on-policy)Backbone=Qwen3-8B, Training Time=5h2026.01 | — | — | 78.3 | — | |
| SDPO (on-policy)Backbone=Olmo3-7B-Instruct, Training Time=1h2026.01 | — | — | 75.3 | — | |
| SDPO (on-policy)Backbone=Olmo3-7B-Instruct, Training Time=5h2026.01 | — | — | 79.2 | — |