Java Method Body Generation on Java suite (test) benchmark leaderboard · SOTA2 Research

Java Method Body Generation on Java suite (test)

99.82Undeclared Variable Access CheckNSG

Evaluation Results

Method	Links
NSG2021.10		99.82	99.55	99.53	99.01	99.69	97.53	98.01	97.96	90.97	97.08	97.1	93.84	100	86.41
CODEGPT2021.10		90.94	—	—	94.37	91.32	97.84	97.83	88.71	85.23	88.1	99.57	97.64	97.08	67.73
GPTNeo1.3B2021.10		90.36	—	—	91.73	90.51	98.09	98.01	86.36	85.09	88.13	99.8	96.46	98.13	64.88
GPTNeo125M2021.10		89.87	—	—	93.9	90.36	98.36	97.38	87.03	84.05	87.25	99.61	96.42	98.18	65.26
CODEX2021.10		88.62	—	—	90.82	88.86	98.35	98.53	92.28	88.13	91.42	98.44	96.82	96.41	47.49
GNN2NAG2021.10		47.44	25.78	15.4	21.2	28.92	21.43	23.86	9.27	12.34	16.31	94.02	20.95	100	17.34
CNG2021.10		19.78	11.03	12.75	21.56	17.92	12.23	16.4	16.09	9.51	13.56	99.92	24.29	100	12.87