DP auto-formulation on DP-Bench
73.3Easy AccuracyDeepSeek-R1
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| DeepSeek-R1Type=Baseline Large-Scale, Parameters=671B2025.07 | 73.3 | 28.6 | 59.1 | 51 | |
| DPLM-7B-SFT-GRPOType=Ours, Parameters=7B2025.07 | 66.7 | 40.5 | 58.3 | 53.6 | |
| DeepSeek-V3Type=Baseline Large-Scale, Parameters=671B2025.07 | 51.1 | 26.2 | 43.2 | 38.7 | |
| GPT-5Type=Baseline Large-Scale, Parameters=*300B2025.07 | 50 | 35.7 | 45.5 | 42.9 | |
| o1Type=Baseline Large-Scale, Parameters=*300B2025.07 | 48.9 | 31 | 43.2 | 39.9 | |
| GPT-4oType=Baseline Large-Scale, Parameters=*200B2025.07 | 47.8 | 19 | 38.6 | 33.4 | |
| Qwen-2.5-72B-InstructType=Baseline Large-Scale, Parameters=72B2025.07 | 43.3 | 21.4 | 36.4 | 32.4 | |
| DPLM-7B-SFT (SFT Only)Type=Ours, Parameters=7B2025.07 | 40 | 21.4 | 34.1 | 30.7 | |
| Qwen-2.5-32B-InstructType=Baseline Large-Scale, Parameters=32B2025.07 | 37.8 | 21.4 | 32.6 | 29.6 | |
| Qwen-2.5-7B-InstructType=Baseline Small-Scale, Parameters=7B2025.07 | 11.1 | 4.8 | 9.1 | 7.9 | |
| Gemma-2-9B-ItType=Baseline Small-Scale, Parameters=9B2025.07 | 6.7 | 2.4 | 5.3 | 4.5 | |
| Llama-3.1-8B-InstructType=Baseline Small-Scale, Parameters=8B2025.07 | 6.7 | 2.4 | 5.3 | 4.5 |