Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

RepoEval dataset and benchmarks · SOTA2 Research

Research Datasets

RepoEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

API Invocation Completion

RepoEval 1.0 (test)

50.13Exact Match

24

Line Completion

RepoEval 1.0 (test)

57.75Exact Match

24

Repository-level code-completion

RepoEval (test)

49.9Exact Match

8

Function Body Completion

RepoEval Function Body Completion (All)

42.63Pass Rate

6

Function Completion

RepoEval leopard (ai-betty)

64.99Pass@5

3

Function Completion

RepoEval deepmind tracr

56.11Pass@5

3

Function Completion

RepoEval amazon-science patchcore-inspection

66.94Pass@5

3

Popular tasks

Function Completion Line Completion Function Body Completion API Invocation Completion Repository-level code completion

Follow for updates

Join SOTA2 Contact