Physics Reasoning on Public Physics Benchmarks Suite (test)
64.4Average AccuracyReason, Reward, Refine
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Reason, Reward, RefineBase Model=LLaMA 3.2 3B2026.07 | 64.4 | — | — | — | |
| Reason, Reward, RefineBase Model=Phi 3.5 Mini 3.8B2026.07 | 61.5 | — | — | — | |
| Reason, Reward, RefineBase Model=Qwen 2.5 1.5B2026.07 | 58.5 | — | — | — | |
| RAGBase Model=Phi 3.5 Mini 3.8B2026.07 | 54.3 | — | — | — | |
| Reason, Reward, RefineBase Model=LLaMA 3.2 1B2026.07 | 53.9 | — | — | — | |
| Logic-DistillBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=40k, SFT Dataset=Ours (Logic-Distill)2026.05 | 53.42 | 53.49 | 53.71 | 53.05 | |
| RAGBase Model=Qwen 2.5 1.5B2026.07 | 53.3 | — | — | — | |
| Reasoning Style TransferBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=80k, SFT Dataset=Ours (RST)2026.05 | 52.26 | 60.46 | 55.95 | 40.36 | |
| DPOBase Model=Phi 3.5 Mini 3.8B2026.07 | 52.1 | — | — | — | |
| RAGBase Model=LLaMA 3.2 3B2026.07 | 51.2 | — | — | — | |
| SFTBase Model=Phi 3.5 Mini 3.8B2026.07 | 51 | — | — | — | |
| CoTBase Model=Phi 3.5 Mini 3.8B2026.07 | 50.5 | — | — | — | |
| SCP-116kBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=80k, SFT Dataset=SCP-116k2026.05 | 47.34 | 56.59 | 52.33 | 33.09 | |
| CoTBase Model=Qwen 2.5 1.5B2026.07 | 46.1 | — | — | — | |
| Logic-DistillBackbone=Qwen2.5-7B-Instruct, Data Scale=40k, SFT Dataset=Ours (Logic-Distill)2026.05 | 45.04 | 43.02 | 49.22 | 42.88 | |
| CoTBase Model=LLaMA 3.2 3B2026.07 | 44.8 | — | — | — | |
| Direct-DistillBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=80k, SFT Dataset=Ours (Direct-Distill)2026.05 | 44.27 | 51.55 | 50.77 | 30.5 | |
| DPOBase Model=LLaMA 3.2 3B2026.07 | 44.1 | — | — | — | |
| SFTBase Model=LLaMA 3.2 3B2026.07 | 44 | — | — | — | |
| MegaScienceBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=80k, SFT Dataset=MegaScience2026.05 | 43.74 | 53.49 | 50.94 | 26.8 | |
| Reasoning Style TransferBackbone=Qwen2.5-7B-Instruct, Data Scale=80k, SFT Dataset=Ours (RST)2026.05 | 41.41 | 47.67 | 38.86 | 37.71 | |
| RAGBase Model=LLaMA 3.2 1B2026.07 | 41.1 | — | — | — | |
| SFTBase Model=Qwen 2.5 1.5B2026.07 | 41 | — | — | — | |
| DPOBase Model=Qwen 2.5 1.5B2026.07 | 40.7 | — | — | — | |
| Direct-DistillBackbone=Llama-3.1-8B, Data Scale=80k, SFT Dataset=Ours (Direct-Distill)2026.05 | 37.98 | 46.9 | 37.82 | 29.21 | |
| SFTBase Model=LLaMA 3.2 1B2026.07 | 37.5 | — | — | — | |
| DPOBase Model=LLaMA 3.2 1B2026.07 | 37.5 | — | — | — | |
| Direct-DistillBackbone=Qwen2.5-7B-Instruct, Data Scale=80k, SFT Dataset=Ours (Direct-Distill)2026.05 | 36.29 | 37.98 | 48.7 | 22.18 | |
| Logic-DistillBackbone=Llama-3.1-8B, Data Scale=40k, SFT Dataset=Ours (Logic-Distill)2026.05 | 35.14 | 39.53 | 35.75 | 30.13 | |
| SCP-116kBackbone=Llama-3.1-8B, Data Scale=80k, SFT Dataset=SCP-116k2026.05 | 35.08 | 43.02 | 32.64 | 29.57 | |
| SCP-116kBackbone=Qwen2.5-7B-Instruct, Data Scale=80k, SFT Dataset=SCP-116k2026.05 | 34.72 | 41.47 | 43.52 | 19.17 | |
| CoTBase Model=LLaMA 3.2 1B2026.07 | 32.5 | — | — | — | |
| Reasoning Style TransferBackbone=Llama-3.1-8B, Data Scale=80k, SFT Dataset=Ours (RST)2026.05 | 30.98 | 40.7 | 27.46 | 24.77 | |
| MegaScienceBackbone=Qwen2.5-7B-Instruct, Data Scale=80k, SFT Dataset=MegaScience2026.05 | 29.39 | 36.82 | 32.12 | 19.22 | |
| NaturalReasoningBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=80k, SFT Dataset=NaturalReasoning2026.05 | 28.03 | 36.82 | 26.94 | 20.33 | |
| NaturalReasoningBackbone=Qwen2.5-7B-Instruct, Data Scale=80k, SFT Dataset=NaturalReasoning2026.05 | 23.71 | 23.26 | 30.05 | 17.81 | |
| Sci-InstructBackbone=DeepSeek-R1-Distill-Qwen-7B, Data Scale=80k, SFT Dataset=Sci-Instruct2026.05 | 20.6 | 34.5 | 7.14 | 20.15 | |
| MegaScienceBackbone=Llama-3.1-8B, Data Scale=80k, SFT Dataset=MegaScience2026.05 | 19.46 | 27.02 | 15.02 | 16.33 | |
| Sci-InstructBackbone=Qwen2.5-7B-Instruct, Data Scale=80k, SFT Dataset=Sci-Instruct2026.05 | 19.22 | 30.62 | 9.84 | 17.19 | |
| Sci-InstructBackbone=Llama-3.1-8B, Data Scale=80k, SFT Dataset=Sci-Instruct2026.05 | 13.58 | 19.77 | 7.25 | 13.72 | |
| NaturalReasoningBackbone=Llama-3.1-8B, Data Scale=80k, SFT Dataset=NaturalReasoning2026.05 | 12.98 | 7.36 | 17.09 | 14.48 |