Text-to-SQL on BIRD (test)
75.63EXXiYan-SQL
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| XiYan-SQL2024.11 | 75.63 | — | — | — | — | — | — | — | — | |
| XiYan-SQL2025.07 | 75.63 | — | 71.41 | — | — | — | — | — | — | |
| CHASE-SQLbackbone=Gemini2024.11 | 74.79 | — | — | — | — | — | — | — | — | |
| CHASE-SQLBackbone=Gemini2025.07 | 74.79 | — | 70.57 | — | — | — | — | — | — | |
| DSAIRbackbone=GPT-4o2024.11 | 74.12 | — | — | — | — | — | — | — | — | |
| DSAIRBackbone=GPT-4o2025.07 | 74.12 | — | 70.13 | — | — | — | — | — | — | |
| ExSLbackbone=granite-34b-code2024.11 | 73.17 | — | — | — | — | — | — | — | — | |
| ExSLBackbone=granite-34b-code2025.07 | 73.17 | — | 71.37 | — | — | — | — | — | — | |
| AskDatabackbone=GPT-4o2024.11 | 72.39 | — | — | — | — | — | — | — | — | |
| OpenSearch-SQLversion=v2, backbone=GPT-4o2024.11 | 72.28 | — | — | — | — | — | — | — | — | |
| OpenSearch-SQLVersion=v2, Backbone=GPT-4o2025.07 | 72.28 | — | 69.36 | — | — | — | — | — | — | |
| Distillerybackbone=GPT-4o2024.11 | 71.83 | — | — | — | — | — | — | — | — | |
| DistilleryBackbone=GPT-4o2025.07 | 71.83 | — | 67.41 | — | — | — | — | — | — | |
| CHESS2024.11 | 71.1 | — | — | — | — | — | — | — | — | |
| CHESS2025.07 | 71.1 | — | 66.53 | — | — | — | — | — | — | |
| Inf-rl-qwenSetting=Original2026.07 | 70.53 | 87.03 | — | — | — | — | — | — | — | |
| Insights AI2024.11 | 70.26 | — | — | — | — | — | — | — | — | |
| Insights AI2025.07 | 70.26 | — | 66.39 | — | — | — | — | — | — | |
| PURPLE + REDbackbone=GPT-4o2024.11 | 70.21 | — | — | — | — | — | — | — | — | |
| PURPLE + REDBackbone=GPT-4o2025.07 | 70.21 | — | 65.62 | — | — | — | — | — | — | |
| MCS-SQL2024.11 | 65.45 | — | — | — | — | — | — | — | — | |
| MCS-SQL2025.07 | 65.45 | — | 61.23 | — | — | — | — | — | — | |
| OmniSQLSetting=Original2026.07 | 63.23 | 74.77 | — | — | — | — | — | — | — | |
| SuperSQL2024.11 | 62.66 | — | — | — | — | — | — | — | — | |
| SuperSQL2025.07 | 62.66 | — | — | — | — | — | — | — | — | |
| SFT CodeSsize=15B2024.11 | 60.37 | — | — | — | — | — | — | — | — | |
| SFT CodeS-15B2025.07 | 60.37 | — | 61.37 | — | — | — | — | — | — | |
| MAC-SQLBackbone=GPT-4, Oracle Schema=false2023.12 | 59.59 | 67.68 | — | — | — | — | — | — | — | |
| TA-SQLbackbone=GPT-42024.11 | 59.14 | — | — | — | — | — | — | — | — | |
| TA-SQLBackbone=GPT-42025.07 | 59.14 | — | — | — | — | — | — | — | — | |
| DAIL-SQL2024.11 | 57.41 | — | — | — | — | — | — | — | — | |
| DAIL-SQLBackbone=GPT-42023.12 | 57.41 | 61.95 | — | — | — | — | — | — | — | |
| DAIL-SQL2025.07 | 57.41 | — | 54.02 | — | — | — | — | — | — | |
| DIN-SQLBackbone=GPT-42023.04 | 55.9 | 59.44 | — | — | — | — | — | — | — | |
| DIN-SQLBackbone=GPT-42023.12 | 55.9 | 59.44 | — | — | — | — | — | — | — | |
| GPT-42023.04 | 54.89 | 60.77 | — | — | — | — | — | — | — | |
| GPT-42023.12 | 54.89 | 60.77 | — | — | — | — | — | — | — | |
| Gemma-3Setting=Original2026.07 | 53.65 | 63.75 | — | — | — | — | — | — | — | |
| Claude-22023.04 | 49.02 | — | — | — | — | — | — | — | — | |
| Claude-22023.12 | 49.02 | — | — | — | — | — | — | — | — | |
| ChatGPT + COT2023.04 | 40.08 | 56.56 | — | — | — | — | — | — | — | |
| ChatGPTStrategy=CoT2023.12 | 40.08 | 56.56 | — | — | — | — | — | — | — | |
| ChatGPT2023.04 | 39.3 | 51.4 | — | — | — | — | — | — | — | |
| Inf-rl-qwenSetting=Expert2026.07 | 38.2 | 44.87 | — | — | — | — | — | 45.84 | 9.81 | |
| Codex2023.04 | 36.47 | 41.6 | — | — | — | — | — | — | — | |
| Palm-22023.04 | 33.04 | — | — | — | — | — | — | — | — | |
| Palm-22023.12 | 33.04 | — | — | — | — | — | — | — | — | |
| OmniSQLSetting=Expert2026.07 | 30.31 | 36.63 | — | — | — | — | — | 52.06 | 7.09 | |
| Gemma-3Setting=Expert2026.07 | 24.64 | 29.38 | — | — | — | — | — | 54.07 | 7.73 | |
| T5-3B2023.04 | 24.05 | 27.8 | — | — | — | — | — | — | — | |
| Inf-rl-qwenSetting=SAGE2026.07 | 21.9 | 26.03 | — | — | — | — | — | 68.95 | 7.27 | |
| T5-Large2023.04 | 20.94 | 25 | — | — | — | — | — | — | — | |
| OmniSQLSetting=SAGE2026.07 | 14.21 | 17.06 | — | — | — | — | — | 77.53 | 5.47 | |
| T5-Base2023.04 | 12.89 | 14.7 | — | — | — | — | — | — | — | |
| Gemma-3Setting=SAGE2026.07 | 8.34 | 9.86 | — | — | — | — | — | 84.45 | 5.08 | |
| ATR (w. Contriever)Retrieval Type=Adaptive2026.04 | — | — | — | 55 | 38.5 | 41.6 | 59.4 | — | — | |
| ATR (w. UAE)Retrieval Type=Adaptive2026.04 | — | — | — | 54.3 | 39.6 | 41.9 | 59.5 | — | — | |
| Contrieverk=32026.04 | — | — | — | 45.2 | 29.9 | 35.1 | 48.8 | — | — | |
| Contrieverk=52026.04 | — | — | — | 50.3 | 32.3 | 37.5 | 54 | — | — | |
| Contrieverk=102026.04 | — | — | — | 53.7 | 36.8 | 40.4 | 58.1 | — | — | |
| GPT-4o2025.07 | — | — | 51.75 | — | — | — | — | — | — | |
| JAR (w. Contriever)k=32026.04 | — | — | — | 49.3 | 36.3 | 37.1 | 55.3 | — | — | |
| JAR (w. Contriever)k=52026.04 | — | — | — | 52.4 | 37 | 37.7 | 56.5 | — | — | |
| JAR (w. Contriever)k=102026.04 | — | — | — | 54 | 38.2 | 39.4 | 58.1 | — | — | |
| JAR (w. UAE)k=32026.04 | — | — | — | 49.6 | 36.3 | 37.2 | 52.9 | — | — | |
| JAR (w. UAE)k=52026.04 | — | — | — | 50.6 | 36.8 | 38 | 54.3 | — | — | |
| JAR (w. UAE)k=102026.04 | — | — | — | 53.6 | 38 | 41.1 | 57.7 | — | — | |
| Murrek=32026.04 | — | — | — | 49.3 | 30.1 | 36.1 | 50.3 | — | — | |
| Murrek=52026.04 | — | — | — | 50.5 | 32.5 | 37 | 52.5 | — | — | |
| Murrek=102026.04 | — | — | — | 53.6 | 37.3 | 41.3 | 58.7 | — | — | |
| Oracle2026.04 | — | — | — | 59.5 | 44.9 | 43.1 | 62.8 | — | — | |
| RankZephyrk=32026.04 | — | — | — | 45.1 | 29.7 | 36 | 48.6 | — | — | |
| RankZephyrk=52026.04 | — | — | — | 51.2 | 33.1 | 37.9 | 54.1 | — | — | |
| RankZephyrk=102026.04 | — | — | — | 54.6 | 37 | 40 | 58.7 | — | — | |
| UAEk=32026.04 | — | — | — | 46.1 | 31.8 | 34.2 | 50.5 | — | — | |
| UAEk=52026.04 | — | — | — | 49.7 | 33.6 | 37.6 | 54.8 | — | — | |
| UAEk=102026.04 | — | — | — | 54 | 38.3 | 41.3 | 59.3 | — | — |