Java Method Body Generation on Java suite (test)
99.82Undeclared Variable Access CheckNSG
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NSG2021.10 | 99.82 | 99.55 | 99.53 | 99.01 | 99.69 | 97.53 | 98.01 | 97.96 | 90.97 | 97.08 | 97.1 | 93.84 | 100 | 86.41 | |
| CODEGPT2021.10 | 90.94 | — | — | 94.37 | 91.32 | 97.84 | 97.83 | 88.71 | 85.23 | 88.1 | 99.57 | 97.64 | 97.08 | 67.73 | |
| GPTNeo1.3BParameters=1.3B2021.10 | 90.36 | — | — | 91.73 | 90.51 | 98.09 | 98.01 | 86.36 | 85.09 | 88.13 | 99.8 | 96.46 | 98.13 | 64.88 | |
| GPTNeo125MParameters=125M2021.10 | 89.87 | — | — | 93.9 | 90.36 | 98.36 | 97.38 | 87.03 | 84.05 | 87.25 | 99.61 | 96.42 | 98.18 | 65.26 | |
| CODEX2021.10 | 88.62 | — | — | 90.82 | 88.86 | 98.35 | 98.53 | 92.28 | 88.13 | 91.42 | 98.44 | 96.82 | 96.41 | 47.49 | |
| GNN2NAG2021.10 | 47.44 | 25.78 | 15.4 | 21.2 | 28.92 | 21.43 | 23.86 | 9.27 | 12.34 | 16.31 | 94.02 | 20.95 | 100 | 17.34 | |
| CNG2021.10 | 19.78 | 11.03 | 12.75 | 21.56 | 17.92 | 12.23 | 16.4 | 16.09 | 9.51 | 13.56 | 99.92 | 24.29 | 100 | 12.87 |