Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

DevEval dataset and benchmarks · SOTA2 Research

Research Datasets

DevEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Developer Knowledge Evaluation

DevEval

61Win Rate

7

Docstring Evaluation

DevEval 183 human-written docstrings

4.938Score

5

DevEval

94.8Solve Rate

4

Repository-level code generation

DevEval

442Inference Time

4

Terminal-related CLI agent task

DevEval

39.74Accuracy

2

Popular tasks

Agentic Coding Repository-level Code Generation Docstring Evaluation Terminal-related CLI agent task Developer Knowledge Evaluation

Follow for updates

Join SOTA2 Contact