General Evaluation on AGIEval
70.22AccuracyDeepSeek-R1 (BF16)
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-R1 (BF16)Model=DeepSeek-R1, Setting=W16A16, Sparsity=0, Evaluation Protocol=Generative2025.12 | 70.22 | |
| SQ-formatModel=DeepSeek-R1, Setting=W(SQ5)A8, Sparsity=0.875, Evaluation Protocol=Generative2025.12 | 69.58 | |
| FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.03 | 56.7 | |
| PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 56.2 | |
| C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.03 | 55.91 | |
| CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 52.67 | |
| NVShardBase Model=Qwen2.5-7B, #P/B=47.55, #AP/B=7.632026.03 | 49.36 | |
| HSA-ULTraining Strategy=Annealing, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.11 | 44.08 | |
| C32A2Base Model=Qwen2.5-1.5B, #P/B=37.62, #AP/B=2.942026.03 | 43.26 | |
| FineRMoEBase Model=Qwen2.5-1.5B, #P/B=5.64, #AP/B=1.852026.03 | 43.14 | |
| PTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.03 | 39.83 | |
| CTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.03 | 38.91 | |
| TRM-MoETraining Strategy=Base, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.11 | 38.64 | |
| NVShardBase Model=Qwen2.5-1.5B, #P/B=9.88, #AP/B=1.782026.03 | 36.78 | |
| HSA-ULTraining Strategy=Base, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.11 | 36.52 | |
| Qwen3Training Strategy=Annealing, Architecture=Dense, Total Params=0.6B, Activated Params=0.6B, Training Tokens=36T2025.11 | 33.58 | |
| Qwen2.5Training Strategy=Annealing, Architecture=Dense, Total Params=0.5B, Activated Params=0.5B, Training Tokens=18T2025.11 | 30.01 | |
| HSA-ULTraining Strategy=Annealing, Architecture=Dense, Total Params=0.5B, Activated Params=0.5B, Training Tokens=4T2025.11 | 29.29 | |
| S16A4Base Model=Qwen2.5-7B, #P/B=7.62, #AP/B=3.342026.03 | 28.94 | |
| C32A2Base Model=Qwen2.5-0.5B, #P/B=10.36, #AP/B=0.952026.03 | 28.31 | |
| FineRMoEBase Model=Qwen2.5-0.5B, #P/B=1.68, #AP/B=0.652026.03 | 28.19 | |
| PTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.03 | 28.07 | |
| CTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.03 | 27.31 | |
| NVShardBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.632026.03 | 26.79 | |
| S16A4Base Model=Qwen2.5-1.5B, #P/B=1.78, #AP/B=0.912026.03 | 25.91 | |
| DUBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.942026.03 | 25.86 | |
| DUBase Model=Qwen2.5-7B, #P/B=47.54, #AP/B=13.322026.03 | 25.65 | |
| DUBase Model=Qwen2.5-1.5B, #P/B=9.87, #AP/B=2.932026.03 | 25.52 | |
| S16A4Base Model=Qwen2.5-0.5B, #P/B=0.63, #AP/B=0.402026.03 | 25.47 |