Text-to-SQL on Spider 1.0 (test)
91.2EM Acc (Overall)SIRIUS-SQL
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SIRIUS-SQLModel=SIRIUS-32B + Gemini-3.1 Pro2026.05 | 91.2 | — | — | — | — | — | |
| SIRIUS-SQLModel=SIRIUS-32B + Qwen3-235B2026.05 | 90.41 | — | — | — | — | — | |
| SIRIUSModel=SIRIUS-32B2026.05 | 89.99 | — | — | — | — | — | |
| SIRIUS-SQLModel=SIRIUS-32B + Qwen3-Coder-30B2026.05 | 89.85 | — | — | — | — | — | |
| OmniSQLModel=OmniSQL-32B2026.05 | 89.8 | — | — | — | — | — | |
| DeepEye-SQLModel=Qwen3-Coder-30B2026.05 | 89.8 | — | — | — | — | — | |
| XiYan-SQLModel=XiYanSQL-32B + GPT-4o2026.05 | 89.65 | — | — | — | — | — | |
| MCS-SQLModel=GPT-4 8K2026.05 | 89.6 | — | — | — | — | — | |
| Arctic-Text2SQL-R1Model=Arctic-Text2SQL-R1-32B2026.05 | 88.7 | — | — | — | — | — | |
| Gemini-3.1 ProModel=Gemini-3.1 Pro2026.05 | 88.02 | — | — | — | — | — | |
| DeepEye-SQLModel=Qwen3-235B2026.05 | 87.61 | — | — | — | — | — | |
| CHASE-SQLModel=Gemini 1.52026.05 | 87.6 | — | — | — | — | — | |
| CHESS+GPT-4Params.=NA2024.10 | 87.2 | — | — | — | — | — | |
| CHESS+GPT-4Params=NA2024.10 | 87.2 | — | — | — | — | — | |
| OpenSearch-SQLModel=GPT-4o2026.05 | 87.1 | — | — | — | — | — | |
| DAIL-SQL+GPT-4Params.=NA2024.10 | 86.6 | — | — | — | — | — | |
| DAIL-SQL+GPT-4Params=NA2024.10 | 86.6 | — | — | — | — | — | |
| DAIL-SQLBase LLM=GPT-42023.12 | 86.6 | — | — | — | — | — | |
| DAIL-SQLModel=GPT-42026.05 | 86.6 | — | — | — | — | — | |
| Qwen3-235BModel=Qwen3-235B2026.05 | 86.21 | — | — | — | — | — | |
| DIN-SQL+GPT-4Params.=NA2024.10 | 85.3 | — | — | — | — | — | |
| DIN-SQL+GPT-4Params=NA2024.10 | 85.3 | — | — | — | — | — | |
| DIN-SQLBase LLM=GPT-42023.12 | 85.3 | — | — | — | — | — | |
| DIN-SQLModel=GPT-42026.05 | 85.3 | — | — | — | — | — | |
| Qwen3-CoderModel=Qwen3-Coder-30B2026.05 | 85.1 | — | — | — | — | — | |
| MSc-SQLParams.=9B2024.10 | 84.7 | — | — | — | — | — | |
| MSc-SQLParams=9B2024.10 | 84.7 | — | — | — | — | — | |
| GPT-4oModel=GPT-4o2026.05 | 83.2 | — | — | — | — | — | |
| MAC-SQLBase LLM=GPT-4 (32k)2023.12 | 82.8 | — | — | — | — | — | |
| C3+ChatGPTParams.=NA2024.10 | 82.3 | — | — | — | — | — | |
| C3+ChatGPTParams=NA2024.10 | 82.3 | — | — | — | — | — | |
| C3Base LLM=ChatGPT2023.12 | 82.3 | — | — | — | — | — | |
| Reasoning-SQLModel=Qwen2.5-Coder-14B2026.05 | 81.43 | — | — | — | — | — | |
| RESDSQLParams.=3B2024.10 | 79.9 | — | — | — | — | — | |
| RESDSQLParams=3B2024.10 | 79.9 | — | — | — | — | — | |
| NatSQL+T5Params.=3B2024.10 | 78 | — | — | — | — | — | |
| NatSQL+T5Params=3B2024.10 | 78 | — | — | — | — | — | |
| DTS-SQL+MistralParams.=7B2024.10 | 77 | — | — | — | — | — | |
| DTS-SQL+MistralParams=7B2024.10 | 77 | — | — | — | — | — | |
| MAC-SQLBase LLM=GPT-3.5-Turbo (16k)2023.12 | 75.53 | — | — | — | — | — | |
| GRAPHIX + PICARD + T5-3BPLM category=Text-to-Text, Backbone=T5-3B, Decoding method=PICARD2023.01 | 74 | — | — | — | — | — | |
| S2SQL + ELECTRAPLM category=Encoder-based, Backbone=ELECTRA2023.01 | 72.1 | — | — | — | — | — | |
| LGESQL + ELECTRAPLM category=Encoder-based, Backbone=ELECTRA2023.01 | 72 | — | — | — | — | — | |
| PICARD + T5-3BPLM category=Text-to-Text, Backbone=T5-3B, Decoding method=PICARD2023.01 | 71.9 | — | — | — | — | — | |
| Full Fine-tuning (All)Model=Mamba 2.8B, Params (%)=100.00, Type=Full Fine-tuning2025.03 | 71.8 | 87.5 | 73.5 | 63.8 | 51.8 | — | |
| RASAT + PICARD + T5-3BPLM category=Text-to-Text, Backbone=T5-3B, Decoding method=PICARD2023.01 | 70.9 | — | — | — | — | — | |
| MAC-SQLBase LLM=SQL-Llama (7B)2023.12 | 70.58 | — | — | — | — | — | |
| SADGAPre-trained Model=GAP2021.11 | 70.1 | — | — | — | — | — | |
| RAT-SQL + GAP MODELBackbone=GAP MODEL2020.12 | 69.7 | — | — | — | — | — | |
| RATSQLPre-trained Model=GAP2021.11 | 69.7 | — | — | — | — | — | |
| SMBOP + GRAPPAPLM category=Encoder-based, Backbone=GRAPPA2023.01 | 69.7 | — | — | — | — | — | |
| RAT-SQL + GraPPaEncoder=GraPPa2020.12 | 69.6 | — | — | — | — | — | |
| RATSQLPre-trained Model=GraPPa2021.11 | 69.6 | — | — | — | — | — | |
| RATSQL + GRAPPAPLM category=Encoder-based, Backbone=GRAPPA2023.01 | 69.6 | — | — | — | — | — | |
| NatSQL + GAPPLM category=Encoder-based, Backbone=GAP2023.01 | 68.7 | — | — | — | — | — | |
| RATSQLPre-trained Model=STRUG2021.11 | 68.4 | — | — | — | — | — | |
| SADGAPre-trained Model=BERT-large2021.11 | 66.7 | — | — | — | — | — | |
| ShadowGNNPre-trained Model=RoBERTa2021.11 | 66.1 | — | — | — | — | — | |
| Full Fine-tuning (S6)Model=Mamba 2.8B, Params (%)=4.44, Type=Full Fine-tuning2025.03 | 65.7 | 81.9 | 68.8 | 58 | 41 | — | |
| RAT-SQL V3 + BERTBackbone=BERT2020.12 | 65.6 | — | — | — | — | — | |
| RATSQLPre-trained Model=BERT-large2021.11 | 65.6 | — | — | — | — | — | |
| RATSQL + BERTPLM category=Encoder-based, Backbone=BERT2023.01 | 65.6 | — | — | — | — | — | |
| DuoRATPre-trained Model=BERT-large2021.11 | 65.4 | — | — | — | — | — | |
| YCSQL + BERTBackbone=BERT2020.12 | 65.3 | — | — | — | — | — | |
| YCSQLPre-trained Model=BERT-large2021.11 | 65.3 | — | — | — | — | — | |
| RAT-SQL + BART EncoderBackbone=BART Encoder2020.12 | 65.1 | — | — | — | — | — | |
| BRIDGE v2 + BERTPLM category=Encoder-based, Backbone=BERT2023.01 | 65 | — | — | — | — | — | |
| State-offset Tuning (h)Model=Mamba 2.8B, Params (%)=0.19, Type=State based2025.03 | 65 | 89.1 | 65.9 | 51.7 | 40.4 | — | |
| ShadowGNN2020.12 | 64.8 | — | — | — | — | — | |
| RATSQL-HPFTPre-trained Model=BERT-large2021.11 | 64.4 | — | — | — | — | — | |
| LoRAModel=Mamba 2.8B, Params (%)=0.38, Type=Parameter based2025.03 | 63.9 | 86.3 | 68.2 | 49.4 | 34.3 | — | |
| State-offset Tuning (y)Model=Mamba 2.8B, Params (%)=0.01, Type=State based2025.03 | 63.1 | 85.9 | 64.1 | 52.3 | 37.3 | — | |
| RAT-SQL V2 + BERTBackbone=BERT2020.12 | 61.9 | — | — | — | — | — | |
| AuxNet + BARTBackbone=BART2020.12 | 61.9 | — | — | — | — | — | |
| SQL-LlamaParameter Count=7B2023.12 | 61.63 | — | — | — | — | — | |
| RyanSQL V2 + BERTBackbone=BERT2020.12 | 60.6 | — | — | — | — | — | |
| BitFitModel=Mamba 2.8B, Params (%)=0.02, Type=Parameter based2025.03 | 59.9 | 82.3 | 60.8 | 52.9 | 31.3 | — | |
| Initial State TuningModel=Mamba 2.8B, Params (%)=0.19, Type=State based2025.03 | 59.7 | 82.3 | 62.3 | 43.7 | 35.5 | — | |
| RATSQL2021.11 | 57.2 | — | — | — | — | — | |
| IRNet v2Pre-trained Model=BERT-base2021.11 | 55 | — | — | — | — | — | |
| IRNetPre-training=BERT2019.09 | 54.7 | — | — | — | — | — | |
| IRNet + BERTBackbone=BERT2020.12 | 54.7 | — | — | — | — | — | |
| GNNPre-trained Model=Bertrand-DR2021.11 | 54.6 | — | — | — | — | — | |
| Editing-Based SQL Query GenerationPre-training=BERT2019.09 | 53.4 | — | — | — | — | — | |
| EditSQL + BERTBackbone=BERT2020.12 | 53.4 | — | — | — | — | — | |
| EditSQLPre-trained Model=BERT-base2021.11 | 53.4 | — | — | — | — | — | |
| GAZP + BERTPLM category=Encoder-based, Backbone=BERT2023.01 | 53.3 | — | — | — | — | — | |
| Prompt TuningModel=Mamba 2.8B, Params (%)=0.01, Type=Prompt based2025.03 | 50.7 | 75.4 | 53.8 | 37.4 | 19.3 | — | |
| IRNet v22021.11 | 48.5 | — | — | — | — | — | |
| Global-GNN2020.12 | 47.4 | — | — | — | — | — | |
| Global-GNN2021.11 | 47.4 | — | — | — | — | — | |
| IRNetPre-training=GloVe2019.09 | 46.7 | — | — | — | — | — | |
| Prefix-TuningModel=Mamba 2.8B, Params (%)=10.82, Type=Prompt based2025.03 | 45.1 | 75 | 45.1 | 32.2 | 13.9 | — | |
| GNN2019.09 | 39.4 | — | — | — | — | — | |
| GNN2021.11 | 39.4 | — | — | — | — | — | |
| Additional-scanModel=Mamba 2.8B, Params (%)=0.28, Type=Parameter based2025.03 | 35 | 62 | 31.9 | 27.4 | 12.1 | — | |
| Editing-Based SQL Query GenerationPre-training=GloVe2019.09 | 32.9 | — | — | — | — | — | |
| SyntaxSQLNetdata augmentation=true, table-aware=true, history-aware=true2018.10 | 27.2 | 48 | 27 | 24.3 | 4.6 | — | |
| SyntaxSQLNetdata augmentation=true2019.09 | 27.2 | — | — | — | — | — | |
| Leerecursive decoding=true2019.09 | 24.3 | — | — | — | — | — |