Text2Cypher generation on Crime
68Q@1 AccuracyStarCoder2-7B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| StarCoder2-7BModel Size=7B, Scaling Budget=n=1, n=5, Query Complexity=Averaged (Easy, Medium, Hard)2026.05 | 68 | 20 | 6 | 48 | 62 | |
| Qwen2.5-7BModel Size=7B, Scaling Budget=n=1, n=5, Query Complexity=Averaged (Easy, Medium, Hard)2026.05 | 65 | 21 | 6 | 44 | 59 | |
| CodeLlama-7BModel Size=7B, Scaling Budget=n=1, n=5, Query Complexity=Averaged (Easy, Medium, Hard)2026.05 | 58 | 19 | 5 | 39 | 53 | |
| DeepSeek-6.7BModel Size=6.7B, Scaling Budget=n=1, n=5, Query Complexity=Averaged (Easy, Medium, Hard)2026.05 | 46 | 18 | 4 | 28 | 42 | |
| CodeLlama-13BModel Size=13B, Scaling Budget=n=1, n=5, Query Complexity=Averaged (Easy, Medium, Hard)2026.05 | 40 | 14 | 5 | 26 | 35 |