# Hardware Verification Testbench Generation on CVDP-ECov Source: [Hardware Verification Testbench Generation on CVDP-ECov]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:40:05.132Z Evaluation of model performance on generating hardware verification testbenches using the CVDP-ECov dataset, measured by coverage pass rates and average coverage scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Hardware Verification Testbench Generation | | Dataset (including any stated split) | CVDP-ECov | | Metric | Coverage Pass Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Coverage Pass Rate](): Higher is better; unit: PERCENT. - [Average Coverage Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Coverage Pass Rate | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-235b-A22b-2507 | 69.9 | Not recorded | 6997eab209c33900fcd76730 | | LLM4Cov | 69.2 | Not recorded | 6997eab209c33900fcd76730 | | LLM4Cov | 67 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-Coder-30B-A3B | 63.9 | Not recorded | 6997eab209c33900fcd76730 | | llama-4-maverick (400B) | 60.2 | Not recorded | 6997eab209c33900fcd76730 | | LLM4Cov | 60.2 | Not recorded | 6997eab209c33900fcd76730 | | Qwen2.5-Coder-32B | 59.3 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-30B-A3B-2507 | 50.6 | Not recorded | 6997eab209c33900fcd76730 | | LLM4Cov | 50.6 | Not recorded | 6997eab209c33900fcd76730 | | LLM4Cov | 48.2 | Not recorded | 6997eab209c33900fcd76730 | | LLM4Cov | 45.5 | Not recorded | 6997eab209c33900fcd76730 | | Qwen2.5-72B | 42.9 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-30B-A3B-2507 | 30.1 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-235b-A22b-2507 | 29.9 | Not recorded | 6997eab209c33900fcd76730 | | CodeV-R1-RL-Qwen-7B | 29.9 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-Coder-30B-A3B | 28.4 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-4B-2507 (Base) | 28.4 | Not recorded | 6997eab209c33900fcd76730 | | VeriCoder-Qwen2.5-14B | 28.2 | Not recorded | 6997eab209c33900fcd76730 | | Qwen2.5-Coder-32B | 23.4 | Not recorded | 6997eab209c33900fcd76730 | | llama-4-maverick (400B) | 23.1 | Not recorded | 6997eab209c33900fcd76730 | | Qwen2.5-Coder-7B | 20.5 | Not recorded | 6997eab209c33900fcd76730 | | VeriCoder-Qwen2.5-14B | 17.1 | Not recorded | 6997eab209c33900fcd76730 | | Qwen2.5-72B | 16.4 | Not recorded | 6997eab209c33900fcd76730 | | VeriReason-Qwen2.5-7B | 15.9 | Not recorded | 6997eab209c33900fcd76730 | | Qwen3-4B-2507 (Base) | 14.9 | Not recorded | 6997eab209c33900fcd76730 | | CodeV-R1-RL-Qwen-7B | 14.5 | Not recorded | 6997eab209c33900fcd76730 | | Qwen2.5-Coder-7B | 11.3 | Not recorded | 6997eab209c33900fcd76730 | | VeriReason-Qwen2.5-7B | 8.9 | Not recorded | 6997eab209c33900fcd76730 | ### Qwen3-235b-A22b-2507 Coverage Pass Rate: 69.9. Result ID: 699b01c75698665ec493c6f3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### LLM4Cov Coverage Pass Rate: 69.2. Result ID: 699b01c75698665ec493c70b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"LLM4Cov","Training Stage":"Stage2"}. This record does not assert independent reproduction. ### LLM4Cov Coverage Pass Rate: 67. Result ID: 699b01c75698665ec493c709. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"LLM4Cov","Training Stage":"Stage1"}. This record does not assert independent reproduction. ### Qwen3-Coder-30B-A3B Coverage Pass Rate: 63.9. Result ID: 699b01c75698665ec493c6fb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"Coding Specialized"}. This record does not assert independent reproduction. ### llama-4-maverick (400B) Coverage Pass Rate: 60.2. Result ID: 699b01c75698665ec493c6f1. [Reporting paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### LLM4Cov Coverage Pass Rate: 60.2. Result ID: 699b01c75698665ec493c707. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"LLM4Cov","Training Stage":"Stage0"}. This record does not assert independent reproduction. ### Qwen2.5-Coder-32B Coverage Pass Rate: 59.3. Result ID: 699b01c75698665ec493c6f9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"Coding Specialized"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-2507 Coverage Pass Rate: 50.6. Result ID: 699b01c75698665ec493c6f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### LLM4Cov Coverage Pass Rate: 50.6. Result ID: 699b01c75698665ec493c70c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"LLM4Cov","Training Stage":"Stage2"}. This record does not assert independent reproduction. ### LLM4Cov Coverage Pass Rate: 48.2. Result ID: 699b01c75698665ec493c70a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"LLM4Cov","Training Stage":"Stage1"}. This record does not assert independent reproduction. ### LLM4Cov Coverage Pass Rate: 45.5. Result ID: 699b01c75698665ec493c708. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"LLM4Cov","Training Stage":"Stage0"}. This record does not assert independent reproduction. ### Qwen2.5-72B Coverage Pass Rate: 42.9. Result ID: 699b01c75698665ec493c6f5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-2507 Coverage Pass Rate: 30.1. Result ID: 699b01c75698665ec493c6f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### Qwen3-235b-A22b-2507 Coverage Pass Rate: 29.9. Result ID: 699b01c75698665ec493c6f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### CodeV-R1-RL-Qwen-7B Coverage Pass Rate: 29.9. Result ID: 699b01c75698665ec493c701. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"Hardware Specific"}. This record does not assert independent reproduction. ### Qwen3-Coder-30B-A3B Coverage Pass Rate: 28.4. Result ID: 699b01c75698665ec493c6fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"Coding Specialized"}. This record does not assert independent reproduction. ### Qwen3-4B-2507 (Base) Coverage Pass Rate: 28.4. Result ID: 699b01c75698665ec493c705. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"LLM4Cov","Training Stage":"Base"}. This record does not assert independent reproduction. ### VeriCoder-Qwen2.5-14B Coverage Pass Rate: 28.2. Result ID: 699b01c75698665ec493c6ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"Hardware Specific"}. This record does not assert independent reproduction. ### Qwen2.5-Coder-32B Coverage Pass Rate: 23.4. Result ID: 699b01c75698665ec493c6fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"Coding Specialized"}. This record does not assert independent reproduction. ### llama-4-maverick (400B) Coverage Pass Rate: 23.1. Result ID: 699b01c75698665ec493c6f2. [Reporting paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### Qwen2.5-Coder-7B Coverage Pass Rate: 20.5. Result ID: 699b01c75698665ec493c6fd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"Coding Specialized"}. This record does not assert independent reproduction. ### VeriCoder-Qwen2.5-14B Coverage Pass Rate: 17.1. Result ID: 699b01c75698665ec493c700. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"Hardware Specific"}. This record does not assert independent reproduction. ### Qwen2.5-72B Coverage Pass Rate: 16.4. Result ID: 699b01c75698665ec493c6f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"General Purpose"}. This record does not assert independent reproduction. ### VeriReason-Qwen2.5-7B Coverage Pass Rate: 15.9. Result ID: 699b01c75698665ec493c703. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Agentic","Model Type":"Hardware Specific"}. This record does not assert independent reproduction. ### Qwen3-4B-2507 (Base) Coverage Pass Rate: 14.9. Result ID: 699b01c75698665ec493c706. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"LLM4Cov","Training Stage":"Base"}. This record does not assert independent reproduction. ### CodeV-R1-RL-Qwen-7B Coverage Pass Rate: 14.5. Result ID: 699b01c75698665ec493c702. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"Hardware Specific"}. This record does not assert independent reproduction. ### Qwen2.5-Coder-7B Coverage Pass Rate: 11.3. Result ID: 699b01c75698665ec493c6fe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"Coding Specialized"}. This record does not assert independent reproduction. ### VeriReason-Qwen2.5-7B Coverage Pass Rate: 8.9. Result ID: 699b01c75698665ec493c704. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Mode":"Direct Inference","Model Type":"Hardware Specific"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.