Text-to-SQL on LogicCat
24.28Exact MatchIESR
Evaluation results
| Method | Links | |
|---|---|---|
| IESRBackbone=XiYanSQL, Size=7B2026.02 | 24.28 | |
| IESRbackbone=XiYanSQL-QwenCoder-7B-25042026.02 | 24.28 | |
| LinkAlignBackbone=XiYanSQL, Size=7B2026.02 | 22.56 | |
| IESRBackbone=OmniSQL, Size=7B2026.02 | 22.41 | |
| IESRbackbone=OmniSQL-7B2026.02 | 22.41 | |
| IESRBackbone=Seed-Coder, Size=8B2026.02 | 22.11 | |
| IESRbackbone=Seed-Coder-8B2026.02 | 22.11 | |
| IESRBackbone=Qwen2.5-Coder, Size=7B2026.02 | 21.66 | |
| IESRbackbone=Qwen2.5-Coder-7B2026.02 | 21.66 | |
| Alpha-SQLBackbone=XiYanSQL, Size=7B2026.02 | 19.78 | |
| LinkAlignBackbone=Qwen2.5-Coder, Size=7B2026.02 | 19.56 | |
| Alpha-SQLBackbone=Seed-Coder, Size=8B2026.02 | 19.34 | |
| CHESS-SQLBackbone=XiYanSQL, Size=7B2026.02 | 19.21 | |
| LinkAlignBackbone=Seed-Coder, Size=8B2026.02 | 18.89 | |
| Alpha-SQLBackbone=Qwen2.5-Coder, Size=7B2026.02 | 18.32 | |
| IESRbackbone=Qwen3-8B2026.02 | 18.23 | |
| SQL-O1Backbone=XiYanSQL, Size=7B, tuning=true2026.02 | 17.91 | |
| SQL-O1Backbone=Seed-Coder, Size=8B, tuning=true2026.02 | 17.33 | |
| Standard COTBackbone=XiYanSQL, Size=7B2026.02 | 16.83 | |
| IESRbackbone=Qwen2.5-7B-Instruct2026.02 | 16.69 | |
| Alpha-SQLBackbone=OmniSQL, Size=7B2026.02 | 16.56 | |
| SQL-O1Backbone=Qwen2.5-Coder, Size=7B, tuning=true2026.02 | 16.42 | |
| CHESS-SQLBackbone=Qwen2.5-Coder, Size=7B2026.02 | 16.16 | |
| o4-mini2026.02 | 14.96 | |
| DTS-SQLBackbone=XiYanSQL, Size=7B2026.02 | 14.88 | |
| Standard COTBackbone=Seed-Coder, Size=8B2026.02 | 14.53 | |
| CHESS-SQLBackbone=Seed-Coder, Size=8B2026.02 | 14.52 | |
| LinkAlignBackbone=OmniSQL, Size=7B2026.02 | 14.34 | |
| DTS-SQLBackbone=Qwen2.5-Coder, Size=7B2026.02 | 14.12 | |
| Claude-3.7-Sonnet2026.02 | 13.99 | |
| GPT-4.12026.02 | 13.57 | |
| SQL-O1Backbone=OmniSQL, Size=7B, tuning=true2026.02 | 13.24 | |
| DTS-SQLBackbone=OmniSQL, Size=7B2026.02 | 12.67 | |
| Din-sqlBackbone=XiYanSQL, Size=7B2026.02 | 11.56 | |
| Standard COTBackbone=OmniSQL, Size=7B2026.02 | 11.56 | |
| Claude-4.0-Sonnet2026.02 | 11.51 | |
| DAIL-sqlBackbone=XiYanSQL, Size=7B2026.02 | 10.98 | |
| CHESS-SQLBackbone=OmniSQL, Size=7B2026.02 | 10.89 | |
| DAIL-sqlBackbone=Qwen2.5-Coder, Size=7B2026.02 | 10.73 | |
| Qwen3-235B-A22B2026.02 | 10.52 | |
| Din-sqlBackbone=Qwen2.5-Coder, Size=7B2026.02 | 10.31 | |
| DTS-SQLBackbone=Seed-Coder, Size=8B2026.02 | 10.31 | |
| GPT-4o2026.02 | 10.17 | |
| Gemini-2.5-Pro2026.02 | 10.17 | |
| Standard COTBackbone=Qwen2.5-Coder, Size=7B2026.02 | 10.07 | |
| DeepSeek-R12026.02 | 9.98 | |
| Din-sqlBackbone=OmniSQL, Size=7B2026.02 | 9.56 | |
| DAIL-sqlBackbone=OmniSQL, Size=7B2026.02 | 9.43 | |
| Qwen3-Coder-480B-A35B-Instruct2026.02 | 9.42 | |
| o3-mini2026.02 | 9.11 | |
| Din-sqlBackbone=Seed-Coder, Size=8B2026.02 | 8.98 | |
| Deepseek-V32026.02 | 8.7 | |
| DAIL-sqlBackbone=Seed-Coder, Size=8B2026.02 | 8.18 | |
| Cot-sqlBackbone=XiYanSQL, Size=7B2026.02 | 7.89 | |
| Cot-sqlBackbone=OmniSQL, Size=7B2026.02 | 6.93 | |
| Cot-sqlBackbone=Seed-Coder, Size=8B2026.02 | 6.58 | |
| Cot-sqlBackbone=Qwen2.5-Coder, Size=7B2026.02 | 5.07 | |
| Seed-Coder-8B-Reasoning2026.02 | 4.16 |