# Code Completion on BigCodeBench Source: [Code Completion on BigCodeBench]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:45:35.398Z Evaluation of model performance on the BigCodeBench dataset for code completion tasks, measured by Full and Hard difficulty scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Code Completion | | Dataset (including any stated split) | BigCodeBench | | Metric | Full Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Full Score](): Higher is better; unit: PERCENT. - [Hard Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Full Score | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-Coder-7B | 45.8 | Not recorded | 68fa77871e209131cea73dc7 | | CodeQwen1.5-7B | 45.6 | Not recorded | 68fa77871e209131cea73dc7 | | Yi-Coder-9B | 42.9 | Not recorded | 68fa77871e209131cea73dc7 | | DS-Coder-6.7B-Base | 41.1 | Not recorded | 68fa77871e209131cea73dc7 | | OpenCoder-8B-Base | 40.5 | Not recorded | 68fa77871e209131cea73dc7 | | StarCoder2-15B | 38.4 | Not recorded | 68fa77871e209131cea73dc7 | | CodeGemma-7B | 38.3 | Not recorded | 68fa77871e209131cea73dc7 | | Qwen2.5-Coder-1.5B | 34.6 | Not recorded | 68fa77871e209131cea73dc7 | | DS-Coder-V2-Lite-Base (MoE) | 30.6 | Not recorded | 68fa77871e209131cea73dc7 | | CodeLlama-7B | 28.7 | Not recorded | 68fa77871e209131cea73dc7 | | StarCoder2-7B | 27.7 | Not recorded | 68fa77871e209131cea73dc7 | | DeepSeek-Coder-1.3B-Base | 26.1 | Not recorded | 68fa77871e209131cea73dc7 | | OpenCoder-1.5B-Base | 24.5 | Not recorded | 68fa77871e209131cea73dc7 | | CodeGemma-2B | 23.9 | Not recorded | 68fa77871e209131cea73dc7 | | Yi-Coder-1.5B | 23.5 | Not recorded | 68fa77871e209131cea73dc7 | | StarCoder2-3B | 21.4 | Not recorded | 68fa77871e209131cea73dc7 | | Crystal-7B | 10.8 | Not recorded | 68fa77871e209131cea73dc7 | ### Qwen2.5-Coder-7B Full Score: 45.8. Result ID: 69953e583badfaffe6277b64. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### CodeQwen1.5-7B Full Score: 45.6. Result ID: 69953e583badfaffe6277b62. [Reporting paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Yi-Coder-9B Full Score: 42.9. Result ID: 69953e583badfaffe6277b63. [Reporting paper]() Recorded evaluation configuration: {"Size":"9B"}. This record does not assert independent reproduction. ### DS-Coder-6.7B-Base Full Score: 41.1. Result ID: 69953e583badfaffe6277b60. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"6.7B"}. This record does not assert independent reproduction. ### OpenCoder-8B-Base Full Score: 40.5. Result ID: 69953e583badfaffe6277b68. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. ### StarCoder2-15B Full Score: 38.4. Result ID: 69953e583badfaffe6277b67. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"15B"}. This record does not assert independent reproduction. ### CodeGemma-7B Full Score: 38.3. Result ID: 69953e583badfaffe6277b5f. [Reporting paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Qwen2.5-Coder-1.5B Full Score: 34.6. Result ID: 69953e583badfaffe6277b5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"1.5B"}. This record does not assert independent reproduction. ### DS-Coder-V2-Lite-Base (MoE) Full Score: 30.6. Result ID: 69953e583badfaffe6277b61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"16B"}. This record does not assert independent reproduction. ### CodeLlama-7B Full Score: 28.7. Result ID: 69953e583badfaffe6277b5e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### StarCoder2-7B Full Score: 27.7. Result ID: 69953e583badfaffe6277b66. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### DeepSeek-Coder-1.3B-Base Full Score: 26.1. Result ID: 69953e583badfaffe6277b58. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"1.3B"}. This record does not assert independent reproduction. ### OpenCoder-1.5B-Base Full Score: 24.5. Result ID: 69953e583badfaffe6277b5d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"1.5B"}. This record does not assert independent reproduction. ### CodeGemma-2B Full Score: 23.9. Result ID: 69953e583badfaffe6277b5a. [Reporting paper]() Recorded evaluation configuration: {"Size":"2B"}. This record does not assert independent reproduction. ### Yi-Coder-1.5B Full Score: 23.5. Result ID: 69953e583badfaffe6277b59. [Reporting paper]() Recorded evaluation configuration: {"Size":"1.5B"}. This record does not assert independent reproduction. ### StarCoder2-3B Full Score: 21.4. Result ID: 69953e583badfaffe6277b5c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"3B"}. This record does not assert independent reproduction. ### Crystal-7B Full Score: 10.8. Result ID: 69953e583badfaffe6277b65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.