VerilogEval
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
VerilogEval v1 (Human)
98.1Pass@1
54
VerilogEval 2.0
84.97Pass@1
40
VerilogEval Machine
97.9Pass@1
37
VerilogEval SR v2
83.3Pass@1
34
VerilogEval CC v2
82.7Pass@1
33
VerilogEval 156 cases (test)
0.78Pass@1
32
VerilogEval v2
89.6Pass@1
27
VerilogEval Machine v1
82.9Pass@1
17
VerilogEval CC v2 (test)
79.1Pass@1
11
VerilogEval SR v2 (test)
77.5Pass@1
11
VerilogEval v2
82.4Pass@1
10
VerilogEval 2.0 (test)
82.47Pass@1
10
VerilogEval v2 (41 filtered problems)
0.805Solved Rate
8
VerilogEval 156 problems
99.4Compilation Rate
5