Text-to-SQL on BIRD (test dev)
48.92Execution Accuracy (EX)MAC-SQL + GPT-4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MAC-SQL + GPT-4knowledge=w/ DELLM2024.02 | 48.92 | 54.78 | |
| DAIL-SQL + GPT-4knowledge=w/ DELLM2024.02 | 45.81 | 51.59 | |
| MAC-SQL + GPT-4knowledge=w/o knowledge2024.02 | 43.65 | 48.07 | |
| DAIL-SQL + GPT-4knowledge=w/o knowledge2024.02 | 40.89 | 45.13 | |
| GPT-4knowledge=w/ DELLM2024.02 | 37.94 | 42.15 | |
| Claude-2knowledge=w/ DELLM2024.02 | 35.53 | 39.71 | |
| GPT-3.5 Turboknowledge=w/ DELLM2024.02 | 33.31 | 36.12 | |
| GPT-4knowledge=w/o knowledge2024.02 | 33.25 | 35.92 | |
| GPT-3.5-Turbo + CoTknowledge=w/ DELLM, prompting=Chain-of-Thought2024.02 | 32.79 | 35.51 | |
| Claude-2knowledge=w/o knowledge2024.02 | 30.05 | 32.97 | |
| GPT-3.5 Turboknowledge=w/o knowledge2024.02 | 27.64 | 28.64 | |
| GPT-3.5-Turbo + CoTknowledge=w/o knowledge, prompting=Chain-of-Thought2024.02 | 27.25 | 29.16 | |
| T5-3Bknowledge=w/ DELLM2024.02 | 16.68 | 20.84 | |
| T5-3Bknowledge=w/o knowledge2024.02 | 10.37 | 13.62 |