# Verilog Generation on VerilogEval v2 (41 filtered problems)

Source: [Verilog Generation on VerilogEval v2 (41 filtered problems)](<https://www.sota2.com/research/sota/verilog-generation-on-verilogeval-v2-41-filtered-problems>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-28T08:00:30.984Z

Evaluation of model performance on generating Verilog code using the specific subset of 41 filtered problems from the VerilogEval v2 dataset, measured using metrics such as Solved rate and Pass@1.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Verilog Generation |
| Dataset (including any stated split) | VerilogEval v2 (41 filtered problems) |
| Metric | Solved Rate |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Solved Rate](<https://www.sota2.com/research/sota/verilog-generation-on-verilogeval-v2-41-filtered-problems.md?metric=Solved%20Rate>): Higher is better; unit: PERCENT.

- [Pass@1](<https://www.sota2.com/research/sota/verilog-generation-on-verilogeval-v2-41-filtered-problems.md?metric=Pass%401>): Higher is better; unit: PERCENT.

- [Gain](<https://www.sota2.com/research/sota/verilog-generation-on-verilogeval-v2-41-filtered-problems.md?metric=Gain>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Solved Rate | Result date | Report paper |
| --- | --- | --- | --- |
| S1G4E3 | 0.8048780487804879 | Not recorded | 69f03eef55a3c32af89f9f29 |
| S2E1E2E3 | 0.8048780487804879 | Not recorded | 69f03eef55a3c32af89f9f29 |
| MAGE | 0.8048780487804879 | Not recorded | 69f03eef55a3c32af89f9f29 |
| verilogcoder | 0.7804878048780488 | Not recorded | 69f03eef55a3c32af89f9f29 |
| S1G4E1E2 | 0.7317073170731707 | Not recorded | 69f03eef55a3c32af89f9f29 |
| hierarchy-verilog | 0.6585365853658537 | Not recorded | 69f03eef55a3c32af89f9f29 |
| S1G4 | 0.5121951219512195 | Not recorded | 69f03eef55a3c32af89f9f29 |
| RTLFixer | 0.4146341463414634 | Not recorded | 69f03eef55a3c32af89f9f29 |

### S1G4E3

Solved Rate: 0.8048780487804879. Result ID: 69f067b847f34ce924d612a2.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

Recorded evaluation configuration: {"loop enabled":"true"}. This record does not assert independent reproduction.

### S2E1E2E3

Solved Rate: 0.8048780487804879. Result ID: 69f067b847f34ce924d612a3.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

Recorded evaluation configuration: {"loop enabled":"true"}. This record does not assert independent reproduction.

### MAGE

Solved Rate: 0.8048780487804879. Result ID: 69f067b847f34ce924d612a6.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76301e209131cea6bbdc.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### verilogcoder

Solved Rate: 0.7804878048780488. Result ID: 69f067b847f34ce924d612a5.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77481e209131cea72380.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### S1G4E1E2

Solved Rate: 0.7317073170731707. Result ID: 69f067b847f34ce924d612a1.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

Recorded evaluation configuration: {"loop enabled":"true"}. This record does not assert independent reproduction.

### hierarchy-verilog

Solved Rate: 0.6585365853658537. Result ID: 69f067b847f34ce924d612a4.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bff1e209131cea8bc49.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### S1G4

Solved Rate: 0.5121951219512195. Result ID: 69f067b847f34ce924d612a0.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### RTLFixer

Solved Rate: 0.4146341463414634. Result ID: 69f067b847f34ce924d612a7.

[Reporting paper](<https://www.sota2.com/research/paper/69f03eef55a3c32af89f9f29.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa734a1e209131cea5b997.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.