Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

DS-1000 dataset and benchmarks · SOTA2 Research

Research Datasets

DS-1000

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Code Generation

DS-1000

69.9Accuracy

35

Code Generation

DS-1000

58.65Pass@1

28

DS-1000

64.34Accuracy

19

Code Generation

DS-1000 1.0 (test)

67.2Matplotlib

19

Code Hallucination Detection

DS-1000

0.95OP

16

Code Generation

DS-1000

68.6Matplotlib Score

15

Data science code generation

DS-1000

60.3Matplotlib Score

13

Data Science Code Completion

DS-1000

32Pandas (Pass@1)

9

DS-1000

56.5Performance Score

8

Code Completion

DS-1000 (test)

56.1Matplotlib Success Rate

8

Code Generation

DS-1000 Python

39.7Pass@1

7

Continuous improvement of language agents

DS-1000 StreamBench (32 held-out episodes)

56.3Pass@4

5

Code Completion

DS-1000 1.0 (test)

55.2Success Rate (Matplotlib)

5

Code Suggestion

DS-1000

51Binary Reward

4

Code Test Generation

DS-1000

68.12Source Pass Rate

4

DS-1000

42.44Pass@1

3

DS-1000 (test)

6.9Accuracy

3

DS-1000 1.0 (test)

55.2Success Rate (Matplotlib)

3

Popular tasks

Code Generation Code Completion Data Science Data science code generation Data Science Code Completion

Follow for updates

Join SOTA2 Contact