Scientific Reasoning on SciKnowEval
82.38Chemistry AccuracyVPD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| VPDBackbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 82.38 | 77.15 | 80.55 | 68 | 77.66 | |
| SDPOBackbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 82.26 | 74.44 | 74.3 | 61.62 | 79.59 | |
| VPDBackbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 81.88 | 74.34 | 73.67 | 64.75 | 77.06 | |
| SDPO+RL (Joint Loss)Backbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 81.01 | 73.5 | 73.12 | 66 | 73.87 | |
| SDPO+RL (Joint Loss)Backbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 80.51 | 69.14 | 69.45 | 55.37 | 71.21 | |
| VPDBackbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 80.33 | 70.8 | 71.17 | 55.62 | 76.06 | |
| SDPOBackbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 80.15 | 66.07 | 63.91 | 50 | 70.21 | |
| RLSDBackbone=Qwen3-8B, Training Steps=752026.05 | 80.1 | — | 69.6 | 56.5 | 76.7 | |
| AMR-SDBackbone=Qwen2.5-7B-Instruct, Training Steps=752026.05 | 77.6 | — | 79.7 | 66.4 | 80.3 | |
| SDPO+RL (Joint Loss)Backbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 77.53 | 66.63 | 62.11 | 58 | 68.88 | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Training Steps=752026.05 | 77.5 | — | 77.7 | 48.2 | 74.2 | |
| RLSDBackbone=Qwen2.5-7B-Instruct, Training Steps=752026.05 | 77.4 | — | 70.6 | 54.6 | 79.9 | |
| SDPOBackbone=Qwen3-8B, Training Steps=752026.05 | 77.2 | — | 72.8 | 56.3 | 74.5 | |
| SDPO+RL (Adv Reweight)Backbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 76.67 | 69.09 | 66.56 | 58.25 | 74.87 | |
| GRPOBackbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 76.58 | 73.11 | 76.09 | 62.5 | 77.26 | |
| GRPOBackbone=Qwen3-8B, Training Steps=752026.05 | 76.5 | — | 74.3 | 60.5 | 79.9 | |
| SDPO+RL (Adv Reweight)Backbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 76.46 | 71.01 | 70.39 | 61.37 | 75.8 | |
| GRPOBackbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 75.65 | 69.81 | 67.42 | 61.25 | 74.93 | |
| AMR-SDBackbone=Qwen3-8B, Training Steps=752026.05 | 75.5 | — | 75.9 | 63.7 | 79.2 | |
| SDPO+RL (Adv Reshape)Backbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 75.18 | 69.1 | 74.14 | 52 | 75.07 | |
| SDPOBackbone=Qwen2.5-7B-Instruct, Training Steps=752026.05 | 74.1 | — | 68.2 | 51.8 | 70.6 | |
| LMSIEval=in2026.07 | 73.72 | 73.98 | 78.54 | 74.59 | 69.08 | |
| N-OPSDEval=cross2026.07 | 73.59 | 72.74 | 71.37 | 72.71 | 73.27 | |
| TTRLEval=cross2026.07 | 73.46 | 72.84 | 71.28 | 73.31 | 73.32 | |
| SDPO+RL (Adv Reweight)Backbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 73.36 | 67.24 | 62.58 | 57.75 | 75.27 | |
| IntuitorEval=in2026.07 | 72.63 | 76.42 | 83.43 | 76.21 | 73.41 | |
| TTRLEval=in2026.07 | 72.43 | 75.92 | 82.98 | 75 | 73.26 | |
| SDPOBackbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 72.41 | 66.34 | 59.92 | 61.5 | 71.54 | |
| Qwen3-4B base2026.07 | 72.11 | 74.35 | 80.2 | 73.98 | 71.12 | |
| N-OPSDEval=in2026.07 | 72.03 | 75.64 | 83.13 | 74.02 | 73.39 | |
| LMSIEval=cross2026.07 | 71.19 | 68.06 | 67.02 | 68.57 | 65.48 | |
| IntuitorEval=cross2026.07 | 70.76 | 70.35 | 68.33 | 69.5 | 72.79 | |
| GRPOBackbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 69.88 | 65.71 | 66.09 | 52.88 | 74 | |
| CREDITModel=OLMo-3-7B-Instruct, Evaluation Protocol=best avg@162026.05 | 69.2 | — | 72.3 | 54.1 | 77.9 | |
| SDPOBackbone=Qwen3-8B, Training Steps=152026.05 | 69.2 | — | 62.4 | 46.9 | 69.2 | |
| SDPOModel=OLMo-3-7B-Instruct, Evaluation Protocol=best avg@162026.05 | 66.8 | — | 71.4 | 42 | 77.2 | |
| CREDITModel=Qwen3-8B, Evaluation Protocol=best avg@162026.05 | 66.4 | — | 71.7 | 52.4 | 74.2 | |
| RLSDBackbone=Qwen3-8B, Training Steps=152026.05 | 66.3 | — | 59.7 | 47.6 | 74.2 | |
| SDPO+RL (Adv Reshape)Backbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 65.83 | 63.64 | 66.02 | 50.25 | 72.47 | |
| SDPOModel=Qwen3-8B, Evaluation Protocol=best avg@162026.05 | 65.6 | — | 71.8 | 51.8 | 72.6 | |
| AMR-SDBackbone=Qwen3-8B, Training Steps=152026.05 | 65 | — | 62.6 | 47 | 74.8 | |
| GRPOBackbone=Qwen3-8B, Training Steps=152026.05 | 64.9 | — | 61.8 | 46.1 | 73.6 | |
| RLSDBackbone=Qwen2.5-7B-Instruct, Training Steps=152026.05 | 64.8 | — | 64.8 | 45.5 | 71.3 | |
| AMR-SDBackbone=Qwen2.5-7B-Instruct, Training Steps=152026.05 | 59.7 | — | 65.3 | 45.5 | 74 | |
| SDPOBackbone=Qwen2.5-7B-Instruct, Training Steps=152026.05 | 59.5 | — | 59.1 | 44.5 | 59.1 | |
| GRPOModel=Qwen3-8B, Evaluation Protocol=best avg@162026.05 | 59.4 | — | 60.8 | 29.4 | 72.5 | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Training Steps=152026.05 | 59.4 | — | 65 | 45 | 70.9 | |
| SDPO+RL (Adv Reshape)Backbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 54.94 | 58.75 | 58.83 | 54.62 | 66.62 | |
| GRPOModel=OLMo-3-7B-Instruct, Evaluation Protocol=best avg@162026.05 | 46.4 | — | 60.3 | 29.3 | 72.7 | |
| QWEN3-8BBackbone=QWEN3-8B, Feedback Source=contrastive sibling rollouts2026.05 | 42.14 | 47.9 | 59.14 | 31.13 | 59.18 | |
| Qwen3-1.7BBackbone=Qwen3-1.7B, Feedback Source=contrastive sibling rollouts2026.05 | 40.95 | 43.64 | 49.92 | 33.63 | 50.07 | |
| Qwen3-8BBackbone=Qwen3-8B, Training Steps=Untrained2026.05 | 40.5 | — | 57.4 | 28.7 | 59.1 | |
| BaseModel=Qwen3-8B, Evaluation Protocol=best avg@162026.05 | 36.2 | — | 59.2 | 25.8 | 58.6 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Training Steps=Untrained2026.05 | 28.6 | — | 55.6 | 29 | 57.5 | |
| Olmo3-7B-InstructBackbone=Olmo3-7B-Instruct, Feedback Source=contrastive sibling rollouts2026.05 | 22.77 | 27.74 | 37.34 | 16.12 | 34.71 | |
| BaseModel=OLMo-3-7B-Instruct, Evaluation Protocol=best avg@162026.05 | 19.9 | — | 37 | 19.1 | 36.5 | |
| AR-OPD2026.06 | — | 68.5 | — | — | — | |
| Base2026.06 | — | 32.12 | — | — | — | |
| Full OPD2026.06 | — | 68 | — | — | — | |
| OPD2026.05 | — | 29.3 | — | — | — | |
| OPDfull-Vvariant=full-V2026.05 | — | 35.1 | — | — | — | |
| OPDtop-krestriction=top-k2026.05 | — | 29.7 | — | — | — | |
| Partial OPD2026.06 | — | 64.2 | — | — | — | |
| SFT2026.06 | — | 47.4 | — | — | — | |
| Student2026.05 | — | 26.1 | — | — | — | |
| vOPDfull-Vvariant=full-V2026.05 | — | 34.7 | — | — | — | |
| vOPDtop-krestriction=top-k2026.05 | — | 33.2 | — | — | — |