NL-to-SQL on Text2Analysis
58.1Execution AccuracyGPT-4
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.10 | 58.1 | |
| Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.10 | 57.2 | |
| Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.10 | 51.7 | |
| GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.10 | 49.8 | |
| Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.10 | 46.5 | |
| Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.10 | 44.6 | |
| GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.10 | 38.2 | |
| Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.10 | 35.8 | |
| Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.10 | 27.3 | |
| GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.10 | 22.7 | |
| Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.10 | 19.9 | |
| Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.10 | 10.3 |