Functional Correctness on HumanEval (p@k and Precision)
71.57Pass@1Dense
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DenseCompression=0%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 71.57 | 85.58 | 89.63 | 100 | |
| SecRL-PruneCompression=10%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 54.79 | 73.18 | 84.21 | 85.3 | |
| PruneNetCompression=10%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 46.43 | 62.08 | 70.34 | 72 | |
| SecRL-PruneCompression=20%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 41.95 | 60.78 | 78.65 | 72.5 | |
| SecRL-PruneCompression=25%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 38.16 | 57.38 | 75.14 | 68.1 | |
| DenseCompression=0%, Backbone=CodeLlama-7B-Python2026.06 | 36.7 | 68.3 | 87.8 | 100 | |
| SecRL-PruneCompression=30%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 34.37 | 53.98 | 71.63 | 63.7 | |
| DenseCompression=0%, Backbone=CodeLlama-7B-Instruct2026.06 | 27.67 | 60.04 | 83.53 | 100 | |
| SecRL-PruneCompression=10%, Backbone=CodeLlama-7B-Python2026.06 | 26.2 | 53.5 | 78.04 | 79.5 | |
| SecRL-PruneCompression=10%, Backbone=CodeLlama-7B-Instruct2026.06 | 21.37 | 46.91 | 72.25 | 80.6 | |
| PruneNetCompression=20%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 21.3 | 38.57 | 51.05 | 43.9 | |
| SecRL-PruneCompression=20%, Backbone=CodeLlama-7B-Python2026.06 | 18.94 | 35.93 | 58.39 | 56.9 | |
| PruneNetCompression=25%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 18.7 | 34.71 | 47.29 | 39.8 | |
| PruneNetCompression=30%, Backbone=Qwen2.5-Coder-7B-Instruct2026.06 | 16.1 | 30.85 | 43.53 | 35.7 | |
| SecRL-PruneCompression=25%, Backbone=CodeLlama-7B-Python2026.06 | 15.32 | 31.58 | 53.68 | 49.7 | |
| SecRL-PruneCompression=20%, Backbone=CodeLlama-7B-Instruct2026.06 | 15.06 | 33.78 | 60.97 | 61.2 | |
| PruneNetCompression=10%, Backbone=CodeLlama-7B-Instruct2026.06 | 14.96 | 33.07 | 49.85 | 56.3 | |
| PruneNetCompression=10%, Backbone=CodeLlama-7B-Python2026.06 | 14.16 | 30.09 | 42.4 | 43.6 | |
| SecRL-PruneCompression=25%, Backbone=CodeLlama-7B-Instruct2026.06 | 11.91 | 27.72 | 55.33 | 51.8 | |
| SecRL-PruneCompression=30%, Backbone=CodeLlama-7B-Python2026.06 | 11.7 | 27.23 | 48.97 | 42.5 | |
| PruneNetCompression=20%, Backbone=CodeLlama-7B-Python2026.06 | 10.55 | 20.77 | 33.1 | 32.3 | |
| SecRL-PruneCompression=30%, Backbone=CodeLlama-7B-Instruct2026.06 | 8.75 | 20.65 | 49.69 | 41.8 | |
| PruneNetCompression=20%, Backbone=CodeLlama-7B-Instruct2026.06 | 7.98 | 20.5 | 37.19 | 35.8 | |
| PruneNetCompression=25%, Backbone=CodeLlama-7B-Python2026.06 | 7.9 | 15.04 | 23.8 | 23.6 | |
| PruneNetCompression=25%, Backbone=CodeLlama-7B-Instruct2026.06 | 5.96 | 14.73 | 29.88 | 27.3 | |
| PruneNetCompression=30%, Backbone=CodeLlama-7B-Python2026.06 | 5.8 | 10 | 17.6 | 16.8 | |
| PruneNetCompression=30%, Backbone=CodeLlama-7B-Instruct2026.06 | 4.2 | 9.7 | 23.85 | 20 |