Text-to-SQL on Archer (dev)
72.1Execution AccuracySOMA-SQL
Evaluation Results
| Method | Links | |
|---|---|---|
| SOMA-SQLBackbone LLM=GPT-52026.06 | 72.1 | |
| SOMA-SQLBackbone LLM=GPT-5.3 Codex2026.06 | 71.2 | |
| AutoLinkBackbone LLM=GPT-5.3 Codex2026.06 | 68.3 | |
| SOMA-SQLBackbone LLM=Gemma-4-31B2026.06 | 65.4 | |
| AutoLinkBackbone LLM=GPT-52026.06 | 64.4 | |
| AutoLinkBackbone LLM=Gemma-4-31B2026.06 | 55.8 | |
| DeepEye-SQLBackbone LLM=GPT-52026.06 | 49 | |
| DeepEye-SQLBackbone LLM=Gemma-4-31B2026.06 | 43.3 | |
| DeepEye-SQLBackbone LLM=GPT-5.3 Codex2026.06 | 41.4 | |
| IESRBackbone=XiYanSQL, Size=7B2026.02 | 37.28 | |
| CHESS-SQLBackbone=XiYanSQL, Size=7B2026.02 | 35.54 | |
| IESRBackbone=Qwen2.5-Coder, Size=7B2026.02 | 35.21 | |
| Din-sqlBackbone=XiYanSQL, Size=7B2026.02 | 34.41 | |
| DTS-SQLBackbone=XiYanSQL, Size=7B2026.02 | 33.17 | |
| IESRBackbone=OmniSQL, Size=7B2026.02 | 33.11 | |
| CHESS-SQLBackbone=Qwen2.5-Coder, Size=7B2026.02 | 32.45 | |
| DAIL-sqlBackbone=XiYanSQL, Size=7B2026.02 | 32.28 | |
| DTS-SQLBackbone=OmniSQL, Size=7B2026.02 | 32.14 | |
| DTS-SQLBackbone=Qwen2.5-Coder, Size=7B2026.02 | 31.88 | |
| CHESS-SQLBackbone=Seed-Coder, Size=8B2026.02 | 31.56 | |
| Din-sqlBackbone=Qwen2.5-Coder, Size=7B2026.02 | 31.45 | |
| LinkAlignBackbone=XiYanSQL, Size=7B2026.02 | 31.28 | |
| Alpha-SQLBackbone=Seed-Coder, Size=8B2026.02 | 31.22 | |
| IESRBackbone=Seed-Coder, Size=8B2026.02 | 30.66 | |
| DTS-SQLBackbone=Seed-Coder, Size=8B2026.02 | 30.33 | |
| LinkAlignBackbone=OmniSQL, Size=7B2026.02 | 30.21 | |
| Alpha-SQLBackbone=XiYanSQL, Size=7B2026.02 | 30.12 | |
| LinkAlignBackbone=Seed-Coder, Size=8B2026.02 | 30.1 | |
| Alpha-SQLBackbone=OmniSQL, Size=7B2026.02 | 29.88 | |
| DAIL-sqlBackbone=Qwen2.5-Coder, Size=7B2026.02 | 29.67 | |
| LinkAlignBackbone=Qwen2.5-Coder, Size=7B2026.02 | 29.21 | |
| Din-sqlBackbone=Seed-Coder, Size=8B2026.02 | 29.13 | |
| SQL-O1Backbone=Seed-Coder, Size=8B, tuning=true2026.02 | 28.78 | |
| Alpha-SQLBackbone=Qwen2.5-Coder, Size=7B2026.02 | 28.32 | |
| SQL-O1Backbone=XiYanSQL, Size=7B, tuning=true2026.02 | 28.22 | |
| DAIL-sqlBackbone=OmniSQL, Size=7B2026.02 | 27.56 | |
| SQL-O1Backbone=OmniSQL, Size=7B, tuning=true2026.02 | 27.41 | |
| CHESS-SQLBackbone=OmniSQL, Size=7B2026.02 | 27.27 | |
| SQL-O1Backbone=Qwen2.5-Coder, Size=7B, tuning=true2026.02 | 26.56 | |
| Din-sqlBackbone=OmniSQL, Size=7B2026.02 | 26.04 | |
| DAIL-sqlBackbone=Seed-Coder, Size=8B2026.02 | 24.12 | |
| Cot-sqlBackbone=Seed-Coder, Size=8B2026.02 | 19.89 | |
| Cot-sqlBackbone=XiYanSQL, Size=7B2026.02 | 18.46 | |
| Cot-sqlBackbone=OmniSQL, Size=7B2026.02 | 18.14 | |
| Cot-sqlBackbone=Qwen2.5-Coder, Size=7B2026.02 | 17.16 |