RepoEval
Benchmarks
TaskDatasetSOTA resultTrendResults
RepoEval 1.0 (test)
50.13Exact Match
24
RepoEval 1.0 (test)
57.75Exact Match
24
RepoEval (test)
49.9Exact Match
8
RepoEval Function Body Completion (All)
42.63Pass Rate
6
RepoEval leopard (ai-betty)
64.99Pass@5
3
RepoEval deepmind tracr
56.11Pass@5
3
RepoEval amazon-science patchcore-inspection
66.94Pass@5
3