Code Generation on HumanEval (Acc, throughput)
98.2AccuracyDense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DenseModel=DeepSeek R1 Distill Qwen 32B2025.07 | 98.2 | 151.52 | |
| ReasonCacheModel=QwQ 32B2025.07 | 97.2 | 193.52 | |
| ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.07 | 96.9 | 246.01 | |
| DenseModel=QwQ 32B2025.07 | 96.8 | 112.31 | |
| ReasonCacheModel=Phi 4 reasoning plus2025.07 | 94.5 | 338.33 | |
| DenseModel=Phi 4 reasoning plus2025.07 | 93.3 | 223.72 | |
| RandomModel=DeepSeek R1 Distill Qwen 32B2025.07 | 62.2 | 247.11 | |
| RandomModel=QwQ 32B2025.07 | 59.8 | 191.68 | |
| RandomModel=Phi 4 reasoning plus2025.07 | 48.2 | 337.75 |