Text-to-SQL on Spider Snow 2.0 (test)
60.51Execution Accuracy (EX)ProSpy
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ProSpyModel=Claude-Opus-4.5, Single Trial=true2026.06 | 60.51 | — | — | |
| ProSpyModel=Claude + GLM, Single Trial=true2026.06 | 51.74 | — | — | |
| APEX-SQLModel=DeepSeek-R1, Single Trial=false2026.06 | 51.01 | — | — | |
| PExALLM=GPT-o32026.04 | 45.7 | 49.5 | 5.55 | |
| Chat2DBLLM=GPT-o32026.04 | 44.1 | — | — | |
| ProSpyModel=Claude + DeepSeek, Single Trial=true2026.06 | 43.51 | — | — | |
| ProSpyModel=DeepSeek V3.2, Single Trial=true2026.06 | 40.77 | — | — | |
| ProSpyModel=GLM-5, Single Trial=true2026.06 | 40.04 | — | — | |
| ReFoRCELLM=GPT-o32026.04 | 36.6 | 39.7 | 5.44 | |
| ReForceModel=GPT-o3, Single Trial=false2026.06 | 35.83 | — | — | |
| DSR-SQLModel=DeepSeek-R1, Single Trial=true2026.06 | 35.28 | — | — | |
| ReForceModel=o4-mini, Single Trial=false2026.06 | 29.8 | — | — | |
| Spider-AgentModel=Claude-4-Sonnet, Single Trial=true2026.06 | 25.78 | — | — | |
| Spider-AgentLLM=GPT-o32026.04 | 25.2 | 27.4 | 5.9 | |
| QAFD-RAG + SQL-AgentLLM=GPT-4o, Generator=SQL-Agent2026.04 | 23.7 | — | — | |
| Spider-AgentLLM=GPT-4o2026.04 | 16.3 | — | — | |
| Spider-AgentModel=DeepSeek-R1, Single Trial=true2026.06 | 10.55 | — | — | |
| CHASE-SQLLLM=GPT-4o2026.04 | 5.92 | — | — | |
| DIN-SQLLLM=GPT-4o2026.04 | 0.18 | — | — | |
| DAIL-SQLLLM=GPT-4o2026.04 | 0 | — | — | |
| CodeSLLM=GPT-4o2026.04 | 0 | — | — |