Scientific Reasoning on SciBench
44.3AccuracyGPT-4o
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4on=695, Iterative Repair Setting=no iterative repair2026.04 | 44.3 | — | |
| AeroTherm-GPTn=695, Iterative Repair Setting=no iterative repair2026.04 | 42.6 | — | |
| GACmode=+ Token-φ2026.05 | 41.2 | — | |
| GACmode=w/o φ (plain SFT)2026.05 | 40.4 | — | |
| HPT2026.05 | 38.7 | — | |
| LUFFY2026.05 | 38.4 | — | |
| KL-ctrl2026.05 | 38.2 | — | |
| MCPO-DAPOBackbone=Qwen3-8B2026.05 | 37.92 | — | |
| CHORDmode=Token-φ, fixed sched.2026.05 | 37.8 | — | |
| GradNorm-ctrl2026.05 | 37.6 | — | |
| SRFT2026.05 | 37.4 | — | |
| Nash-MTL2026.05 | 37.3 | — | |
| CAGrad2026.05 | 37 | — | |
| MCPO-GRPOBackbone=Qwen3-8B2026.05 | 36.83 | — | |
| MGDA2026.05 | 36.5 | — | |
| DAPOBackbone=Qwen3-8B2026.05 | 36.05 | — | |
| MCPO-DAPOBackbone=Qwen3-4B2026.05 | 35.81 | — | |
| Llama-3-70Bn=695, Iterative Repair Setting=no iterative repair2026.04 | 35.8 | — | |
| MT-GRPOBackbone=Qwen3-8B2026.05 | 35.71 | — | |
| MGSBackbone=Qwen3-8B2026.05 | 35.24 | — | |
| SFT-best + RL2026.05 | 35.1 | — | |
| MCPO-GRPOBackbone=Qwen3-4B2026.05 | 34.92 | — | |
| GRPOBackbone=Qwen3-8B2026.05 | 34.91 | — | |
| DAPOBackbone=Qwen3-4B2026.05 | 34.86 | — | |
| MT-GRPOBackbone=Qwen3-4B2026.05 | 34.52 | — | |
| CLIPOBackbone=Qwen3-8B2026.05 | 34.32 | — | |
| MGSBackbone=Qwen3-4B2026.05 | 34.18 | — | |
| CLIPOBackbone=Qwen3-4B2026.05 | 33.2 | — | |
| GRPOBackbone=Qwen3-4B2026.05 | 33.04 | — | |
| SFT-best2026.05 | 32.4 | — | |
| Qwen2.5-7B-Inst.2026.05 | 28.7 | — | |
| Base ModelBackbone=Qwen3-8B2026.05 | 28.46 | — | |
| Base ModelBackbone=Qwen3-4B2026.05 | 27.01 | — | |
| ChatGLM2-6BParameter Scale=6B~7B2024.01 | — | 1.54 | |
| ChatGLM2-6B-BaseParameter Scale=6B~7B2024.01 | — | 1.2 | |
| ChatGLM3-32B-BaseParameter Scale=30B~32B2024.01 | — | 4.29 | |
| ChatGLM3-6BParameter Scale=6B~7B2024.01 | — | 2.4 | |
| ChatGLM3-6B-BaseParameter Scale=6B~7B2024.01 | — | 2.4 | |
| GPT-3.5-turboParameter Scale=API2024.01 | — | 12.17 | |
| GPT-4Parameter Scale=API2024.01 | — | 28.52 | |
| LLaMA-2-13BParameter Scale=12B~13B2024.01 | — | 1.37 | |
| LLaMA-2-7BParameter Scale=6B~7B2024.01 | — | 0.4 | |
| Llama3-8B-InstructEvaluation Protocol=zero-shot, Parameter Scale=6B~7B2024.01 | — | 1.03 | |
| Llama3-8B-InstructEvaluation Protocol=few-shot, Parameter Scale=6B~7B2024.01 | — | 3.6 | |
| Llama3-8B-Instruct + SciInstructFine-tuning=SciInstruct, Parameter Scale=6B~7B2024.01 | — | 3.6 | |
| Mistral-7B: MetaMATHEvaluation Protocol=zero-shot, Parameter Scale=6B~7B2024.01 | — | 4.63 | |
| Mistral-7B: MetaMATHEvaluation Protocol=few-shot, Parameter Scale=6B~7B2024.01 | — | 6.17 | |
| Mistral-7B: MetaMATH + SciInstructFine-tuning=SciInstruct, Parameter Scale=6B~7B2024.01 | — | 6.23 | |
| SciGLMBackbone=ChatGLM3-6B-Base, Parameter Scale=6B~7B2024.01 | — | 3.77 | |
| SciGLMBackbone=ChatGLM3-32B-Base, Parameter Scale=30B~32B2024.01 | — | 5.15 |