# Code Generation on HumanEval (Normalized Spearman distance) Source: [Code Generation on HumanEval (Normalized Spearman distance)]() Collection index: [llms.txt]() Data updated: 2026-06-17T07:18:19.380Z Evaluation of model performance on the HumanEval dataset for code generation, specifically reporting the Normalized Spearman distance. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Code Generation | | Dataset (including any stated split) | HumanEval | | Metric | Normalized Spearman Distance | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Normalized Spearman Distance](): Higher is better; unit: SCALAR. ## Recorded results | Method | Normalized Spearman Distance | Result date | Report paper | | --- | --- | --- | --- | | Block top-1 | 100 | Not recorded | 6a3229ffd33dfb23e783ea4a | | SingleGraph1 | 96 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Block top-2 | 95 | Not recorded | 6a3229ffd33dfb23e783ea4a | | ScoreWin80 | 93 | Not recorded | 6a3229ffd33dfb23e783ea4a | | NoTrunc boot | 92 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Random50→40×10 | 92 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Trunc50→40 | 82 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Block top-2 | 74 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Block top-1 | 70 | Not recorded | 6a3229ffd33dfb23e783ea4a | | NoTrunc boot | 66 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Random50→20×10 | 66 | Not recorded | 6a3229ffd33dfb23e783ea4a | | ScoreWin40 | 65 | Not recorded | 6a3229ffd33dfb23e783ea4a | | SingleGraph1/all10 | 60 | Not recorded | 6a3229ffd33dfb23e783ea4a | | Trunc25→20 | 57 | Not recorded | 6a3229ffd33dfb23e783ea4a | ### Block top-1 Normalized Spearman Distance: 100. Result ID: 6a324a23fbc373592c2f6371. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### SingleGraph1 Normalized Spearman Distance: 96. Result ID: 6a324a23fbc373592c2f6375. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### Block top-2 Normalized Spearman Distance: 95. Result ID: 6a324a23fbc373592c2f6370. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### ScoreWin80 Normalized Spearman Distance: 93. Result ID: 6a324a23fbc373592c2f6373. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### NoTrunc boot Normalized Spearman Distance: 92. Result ID: 6a324a23fbc373592c2f6372. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### Random50→40×10 Normalized Spearman Distance: 92. Result ID: 6a324a23fbc373592c2f6374. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### Trunc50→40 Normalized Spearman Distance: 82. Result ID: 6a324a23fbc373592c2f636f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Task-prompt"}. This record does not assert independent reproduction. ### Block top-2 Normalized Spearman Distance: 74. Result ID: 6a324a23fbc373592c2f6377. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. ### Block top-1 Normalized Spearman Distance: 70. Result ID: 6a324a23fbc373592c2f6378. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. ### NoTrunc boot Normalized Spearman Distance: 66. Result ID: 6a324a23fbc373592c2f6379. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. ### Random50→20×10 Normalized Spearman Distance: 66. Result ID: 6a324a23fbc373592c2f637b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. ### ScoreWin40 Normalized Spearman Distance: 65. Result ID: 6a324a23fbc373592c2f637a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. ### SingleGraph1/all10 Normalized Spearman Distance: 60. Result ID: 6a324a23fbc373592c2f637c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. ### Trunc25→20 Normalized Spearman Distance: 57. Result ID: 6a324a23fbc373592c2f6376. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Perturbation Level":"Judge-prompt with Prometheus"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.