RTL code understanding on RTL code understanding (test) benchmark leaderboard · SOTA2 Research

RTL code understanding on RTL code understanding (test)

13.96BLEU-4DeepRTL2

Evaluation Results

Method	Links
DeepRTL22025.05		13.96	37.93	20.73	34.34	34.74	82	61.6
DeepRTL22025.05		13.84	37.97	20.69	34.42	34.75	81.3	60.3
DeepRTL22025.05		13.53	37.52	19.68	34.68	33.28	81.4	61.2
DeepRTL22025.05		13.34	37.74	19.54	34.76	33.46	79.8	59.4
DeepRTL2025.05		13.06	37.56	19.85	34.72	34.37	80.6	60
DeepRTL2025.05		12.85	37.43	19.34	34.63	33.09	80.2	59.7
DeepRTL22025.05		12.07	36.37	17.78	33.78	28.56	76.7	60.2
DeepRTL22025.05		11.28	34.29	16.35	33.63	27.73	75.4	58
GPT-4o2025.05		4.59	29.26	11.48	25.74	22.78	76.1	54.9
o1-preview2025.05		3.73	28	10.39	24.98	20.48	74.8	53.5
GPT-3.52025.05		3.34	28.2	10.46	25.11	20.36	74	51
CodeV-DeepSeek2025.05		3.05	25.14	9.78	23.25	20.23	70.5	49.5
CodeV-CodeQwen2025.05		2.8	24.91	8.27	22.75	21.07	74.7	49.9
Llama-3.12025.05		2.68	25.37	10.39	23.75	17.16	73	43
DeepSeek-Coder2025.05		2.56	24.52	7.72	22.45	22.83	75.6	57.1