Repository-level Method Body Completion on Defect4J benchmark leaderboard · SOTA2 Research

Repository-level Method Body Completion on Defect4J

67.8BLEURAMBO Oracle

Evaluation Results

Method	Links
RAMBO Oracle2024.09		67.8	67.96	76.96	61.76	35.29
RAMBO2024.09		66.29	67.82	76.47	63.73	32.84
RawRAG2024.09		66.09	67.36	75	61.27	31.28
RLCoder2024.09		65.78	66.67	75.98	61.76	32.84
RepoCoder2024.09		63.52	65.39	74.02	59.8	31.86
In-File2024.09		63.16	65.66	76.96	60.78	26.96
RepoCoder Oracle2024.09		62.76	64.85	72.06	57.84	31.37
RAMBO Oracle2024.09		54.19	55.9	64.71	46.08	19.12
RepoCoder Oracle2024.09		53.69	54.47	64.71	46.57	19.61
RAMBO2024.09		52.04	54.98	63.73	44.12	20.59
RAMBO Oracle2024.09		51.31	53.54	67.16	49.51	21.57
RepoCoder2024.09		51.27	53.46	60.78	46.57	20.1
RAMBO2024.09		51.03	53.05	69.61	46.57	23.04
RepoCoder Oracle2024.09		50.74	52.96	64.71	45.59	21.57
RLCoder2024.09		50.37	51.12	60.29	42.65	17.54
RawRAG2024.09		48.88	50.93	60.29	45.1	18.01
RepoCoder2024.09		48.39	51.01	60.78	46.08	20.59
RLCoder2024.09		45.4	48.1	57.84	45.59	18.01
RawRAG2024.09		44.96	48.47	61.27	45.1	18.01
In-File2024.09		44.06	46.73	55.88	33.82	8.82
In-File2024.09		39.18	43.24	52.45	34.8	11.27