Natural Language Code Search on RTL code search benchmark
66.2F1 ScoreUniRTL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UniRTLDesign Format=code & graph2026.05 | 66.2 | 65 | 69.2 | |
| UniRTL (w/o graph)Design Format=code2026.05 | 64.4 | 63 | 68.3 | |
| GraphCodeBERTDesign Format=code & graph2026.05 | 63.4 | 61.6 | 67.5 | |
| DeepRTL2-LlamaDesign Format=code2026.05 | 57.2 | 55.7 | 60.8 | |
| CircuitFusionDesign Format=code & graph2026.05 | 56 | 54.2 | 60.8 | |
| DeepRTL2-DeepSeekDesign Format=code2026.05 | 54.7 | 53.2 | 59.2 | |
| DeepRTL2Backbone=Llama, Hard Negatives=False2025.05 | 47.6 | — | — | |
| DeepRTL2Backbone=DeepSeek, Hard Negatives=False2025.05 | 46.4 | — | — | |
| DeepRTL2Backbone=Llama2025.05 | 46.3 | — | — | |
| DeepRTL2Backbone=DeepSeek2025.05 | 45.3 | — | — | |
| GritLM-7BDesign Format=code2026.05 | 41.4 | 39.3 | 47.5 | |
| NV-Embed-v2Design Format=code2026.05 | 38.9 | 36.7 | 45 | |
| text-embedding-3-largeDesign Format=code2026.05 | 37.5 | 35 | 44.2 | |
| text-embedding-3-large2025.05 | 29 | — | — | |
| text-embedding-3-smallDesign Format=code2026.05 | 27.7 | 25.4 | 35 | |
| GritLMParameters=7B2025.05 | 26.9 | — | — | |
| text-embedding-3-small2025.05 | 18.9 | — | — |