# Code Generation on CodeContests+

Source: [Code Generation on CodeContests+](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-13T07:33:36.927Z

Evaluation of Code Generation performance using the CodeContests+ dataset, reporting metrics including LCBv6, HumanEval, MBPP, and BCB.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Code Generation |
| Dataset (including any stated split) | CodeContests+ |
| Metric | LCBv6 Score |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [LCBv6 Score](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=LCBv6%20Score>): Higher is better; unit: PERCENT.

- [HumanEval Score](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=HumanEval%20Score>): Higher is better; unit: PERCENT.

- [MBPP Score](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=MBPP%20Score>): Higher is better; unit: PERCENT.

- [BCB Score](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=BCB%20Score>): Higher is better; unit: PERCENT.

- [Average Pass@1 Score](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=Average%20Pass%401%20Score>): Higher is better; unit: PERCENT.

- [Training Time (hours)](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=Training%20Time%20(hours)>): Lower is better; unit: TIME.

- [Time Change (%)](<https://www.sota2.com/research/sota/code-generation-on-codecontests-plus.md?metric=Time%20Change%20(%25)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | LCBv6 Score | Result date | Report paper |
| --- | --- | --- | --- |
| METIS | 39.1 | Not recorded | 6a04057ddbc836b87d827475 |
| No Curr. | 38.9 | Not recorded | 6a04057ddbc836b87d827475 |
| PCL | 38.3 | Not recorded | 6a04057ddbc836b87d827475 |
| SEC | 37.3 | Not recorded | 6a04057ddbc836b87d827475 |
| ADCL | 37.2 | Not recorded | 6a04057ddbc836b87d827475 |
| PCL | 31.7 | Not recorded | 6a04057ddbc836b87d827475 |
| METIS | 31.7 | Not recorded | 6a04057ddbc836b87d827475 |
| No Curr. | 31.6 | Not recorded | 6a04057ddbc836b87d827475 |
| SEC | 31.1 | Not recorded | 6a04057ddbc836b87d827475 |
| ADCL | 30.7 | Not recorded | 6a04057ddbc836b87d827475 |
| METIS | 19.1 | Not recorded | 6a04057ddbc836b87d827475 |
| PCL | 18.4 | Not recorded | 6a04057ddbc836b87d827475 |
| ADCL | 17.5 | Not recorded | 6a04057ddbc836b87d827475 |
| No Curr. | 17.2 | Not recorded | 6a04057ddbc836b87d827475 |
| SEC | 15.4 | Not recorded | 6a04057ddbc836b87d827475 |

### METIS

LCBv6 Score: 39.1. Result ID: 6a0428e347f34ce924e9b4d5.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

Recorded evaluation configuration: {"Model":"DS-R1-Distill Llama-8B"}. This record does not assert independent reproduction.

### No Curr.

LCBv6 Score: 38.9. Result ID: 6a0428e347f34ce924e9b4d1.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

Recorded evaluation configuration: {"Model":"DS-R1-Distill Llama-8B"}. This record does not assert independent reproduction.

### PCL

LCBv6 Score: 38.3. Result ID: 6a0428e347f34ce924e9b4d4.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bc51e209131cea8a585.md>)

Recorded evaluation configuration: {"Model":"DS-R1-Distill Llama-8B"}. This record does not assert independent reproduction.

### SEC

LCBv6 Score: 37.3. Result ID: 6a0428e347f34ce924e9b4d2.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b461e209131cea87747.md>)

Recorded evaluation configuration: {"Model":"DS-R1-Distill Llama-8B"}. This record does not assert independent reproduction.

### ADCL

LCBv6 Score: 37.2. Result ID: 6a0428e347f34ce924e9b4d3.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b681e209131cea883cb.md>)

Recorded evaluation configuration: {"Model":"DS-R1-Distill Llama-8B"}. This record does not assert independent reproduction.

### PCL

LCBv6 Score: 31.7. Result ID: 6a0428e347f34ce924e9b4ca.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bc51e209131cea8a585.md>)

Recorded evaluation configuration: {"Model":"Qwen3-8B Base"}. This record does not assert independent reproduction.

### METIS

LCBv6 Score: 31.7. Result ID: 6a0428e347f34ce924e9b4cb.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

Recorded evaluation configuration: {"Model":"Qwen3-8B Base"}. This record does not assert independent reproduction.

### No Curr.

LCBv6 Score: 31.6. Result ID: 6a0428e347f34ce924e9b4c7.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

Recorded evaluation configuration: {"Model":"Qwen3-8B Base"}. This record does not assert independent reproduction.

### SEC

LCBv6 Score: 31.1. Result ID: 6a0428e347f34ce924e9b4c8.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b461e209131cea87747.md>)

Recorded evaluation configuration: {"Model":"Qwen3-8B Base"}. This record does not assert independent reproduction.

### ADCL

LCBv6 Score: 30.7. Result ID: 6a0428e347f34ce924e9b4c9.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b681e209131cea883cb.md>)

Recorded evaluation configuration: {"Model":"Qwen3-8B Base"}. This record does not assert independent reproduction.

### METIS

LCBv6 Score: 19.1. Result ID: 6a0428e347f34ce924e9b4d0.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

Recorded evaluation configuration: {"Model":"Llama-3.1 8B-Instruct"}. This record does not assert independent reproduction.

### PCL

LCBv6 Score: 18.4. Result ID: 6a0428e347f34ce924e9b4cf.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bc51e209131cea8a585.md>)

Recorded evaluation configuration: {"Model":"Llama-3.1 8B-Instruct"}. This record does not assert independent reproduction.

### ADCL

LCBv6 Score: 17.5. Result ID: 6a0428e347f34ce924e9b4ce.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b681e209131cea883cb.md>)

Recorded evaluation configuration: {"Model":"Llama-3.1 8B-Instruct"}. This record does not assert independent reproduction.

### No Curr.

LCBv6 Score: 17.2. Result ID: 6a0428e347f34ce924e9b4cc.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

Recorded evaluation configuration: {"Model":"Llama-3.1 8B-Instruct"}. This record does not assert independent reproduction.

### SEC

LCBv6 Score: 15.4. Result ID: 6a0428e347f34ce924e9b4cd.

[Reporting paper](<https://www.sota2.com/research/paper/6a04057ddbc836b87d827475.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b461e209131cea87747.md>)

Recorded evaluation configuration: {"Model":"Llama-3.1 8B-Instruct"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
