NL-to-SQL on WikiSQL
87.4Execution AccuracyTable-Specialist
Evaluation Results
| Method | Links | |
|---|---|---|
| Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.10 | 87.4 | |
| GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.10 | 86.9 | |
| Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.10 | 85.5 | |
| Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.10 | 82.7 | |
| GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.10 | 82.3 | |
| Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.10 | 81.6 | |
| Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.10 | 81.5 | |
| Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.10 | 79.4 | |
| Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.10 | 77.6 | |
| GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.10 | 75.9 | |
| GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.10 | 74.5 | |
| Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.10 | 72.8 | |
| GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.10 | 56.7 | |
| Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.10 | 52.5 | |
| GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.10 | 51 | |
| Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.10 | 40.9 | |
| Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.10 | 35.9 | |
| Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.10 | 33.1 |