# Code Summarization on Code Summarization dataset

Source: [Code Summarization on Code Summarization dataset](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:47:13.797Z

Evaluation of Code Summarization performance on the 'Code Summarization dataset', measured using metrics including IT, OT, TT, and BLEU scores.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Code Summarization |
| Dataset (including any stated split) | Code Summarization dataset |
| Metric | Input Tokens |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Input Tokens](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Input%20Tokens>): Higher is better; unit: SCALAR.

- [Ratio Input Tokens](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Ratio%20Input%20Tokens>): Higher is better; unit: PERCENT.

- [Output Tokens](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Output%20Tokens>): Higher is better; unit: SCALAR.

- [Ratio Output Tokens](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Ratio%20Output%20Tokens>): Higher is better; unit: PERCENT.

- [Total Tokens](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Total%20Tokens>): Higher is better; unit: SCALAR.

- [Ratio Total Tokens](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Ratio%20Total%20Tokens>): Higher is better; unit: PERCENT.

- [BLEU Score](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=BLEU%20Score>): Higher is better; unit: SCALAR.

- [Ratio BLEU](<https://www.sota2.com/research/sota/code-summarization-on-code-summarization-dataset.md?metric=Ratio%20BLEU>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Input Tokens | Result date | Report paper |
| --- | --- | --- | --- |
| Base | 78246 | Not recorded | 68fa793e1e209131cea7cd46 |
| DietCode | 75217 | Not recorded | 68fa793e1e209131cea7cd46 |
| LeanCode | 74938 | Not recorded | 68fa793e1e209131cea7cd46 |
| SlimCode | 74912 | Not recorded | 68fa793e1e209131cea7cd46 |
| DietCode | 72245 | Not recorded | 68fa793e1e209131cea7cd46 |
| SlimCode | 70734 | Not recorded | 68fa793e1e209131cea7cd46 |
| LeanCode | 70207 | Not recorded | 68fa793e1e209131cea7cd46 |
| DietCode | 66844 | Not recorded | 68fa793e1e209131cea7cd46 |
| SlimCode | 65648 | Not recorded | 68fa793e1e209131cea7cd46 |
| LeanCode | 65322 | Not recorded | 68fa793e1e209131cea7cd46 |
| DietCode | 61534 | Not recorded | 68fa793e1e209131cea7cd46 |
| SlimCode | 60585 | Not recorded | 68fa793e1e209131cea7cd46 |
| LeanCode | 60296 | Not recorded | 68fa793e1e209131cea7cd46 |
| DietCode | 56162 | Not recorded | 68fa793e1e209131cea7cd46 |
| SlimCode | 55335 | Not recorded | 68fa793e1e209131cea7cd46 |
| LeanCode | 55261 | Not recorded | 68fa793e1e209131cea7cd46 |

### Base

Input Tokens: 78246. Result ID: 6995d5af3badfaffe6db604f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

Recorded evaluation configuration: {"Removal percentage":"0%"}. This record does not assert independent reproduction.

### DietCode

Input Tokens: 75217. Result ID: 6995d5af3badfaffe6db6050.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6df71e209131cea3d030.md>)

Recorded evaluation configuration: {"Removal percentage":"10%"}. This record does not assert independent reproduction.

### LeanCode

Input Tokens: 74938. Result ID: 6995d5af3badfaffe6db605a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

Recorded evaluation configuration: {"Removal percentage":"10%"}. This record does not assert independent reproduction.

### SlimCode

Input Tokens: 74912. Result ID: 6995d5af3badfaffe6db6055.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b845.md>)

Recorded evaluation configuration: {"Removal percentage":"10%"}. This record does not assert independent reproduction.

### DietCode

Input Tokens: 72245. Result ID: 6995d5af3badfaffe6db6051.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6df71e209131cea3d030.md>)

Recorded evaluation configuration: {"Removal percentage":"20%"}. This record does not assert independent reproduction.

### SlimCode

Input Tokens: 70734. Result ID: 6995d5af3badfaffe6db6056.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b845.md>)

Recorded evaluation configuration: {"Removal percentage":"20%"}. This record does not assert independent reproduction.

### LeanCode

Input Tokens: 70207. Result ID: 6995d5af3badfaffe6db605b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

Recorded evaluation configuration: {"Removal percentage":"20%"}. This record does not assert independent reproduction.

### DietCode

Input Tokens: 66844. Result ID: 6995d5af3badfaffe6db6052.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6df71e209131cea3d030.md>)

Recorded evaluation configuration: {"Removal percentage":"30%"}. This record does not assert independent reproduction.

### SlimCode

Input Tokens: 65648. Result ID: 6995d5af3badfaffe6db6057.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b845.md>)

Recorded evaluation configuration: {"Removal percentage":"30%"}. This record does not assert independent reproduction.

### LeanCode

Input Tokens: 65322. Result ID: 6995d5af3badfaffe6db605c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

Recorded evaluation configuration: {"Removal percentage":"30%"}. This record does not assert independent reproduction.

### DietCode

Input Tokens: 61534. Result ID: 6995d5af3badfaffe6db6053.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6df71e209131cea3d030.md>)

Recorded evaluation configuration: {"Removal percentage":"40%"}. This record does not assert independent reproduction.

### SlimCode

Input Tokens: 60585. Result ID: 6995d5af3badfaffe6db6058.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b845.md>)

Recorded evaluation configuration: {"Removal percentage":"40%"}. This record does not assert independent reproduction.

### LeanCode

Input Tokens: 60296. Result ID: 6995d5af3badfaffe6db605d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

Recorded evaluation configuration: {"Removal percentage":"40%"}. This record does not assert independent reproduction.

### DietCode

Input Tokens: 56162. Result ID: 6995d5af3badfaffe6db6054.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6df71e209131cea3d030.md>)

Recorded evaluation configuration: {"Removal percentage":"50%"}. This record does not assert independent reproduction.

### SlimCode

Input Tokens: 55335. Result ID: 6995d5af3badfaffe6db6059.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b845.md>)

Recorded evaluation configuration: {"Removal percentage":"50%"}. This record does not assert independent reproduction.

### LeanCode

Input Tokens: 55261. Result ID: 6995d5af3badfaffe6db605e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd46.md>)

Recorded evaluation configuration: {"Removal percentage":"50%"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.