Text-to-SQL on Spider schema presentation perturbations (dev)
95.7AST Similarity (Schema)Gemini 3 Pro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini 3 Pro2026.04 | 95.7 | 1.06 | 4.3 | 6.5 | |
| Claude Sonnet 4.52026.04 | 95.5 | 1.13 | 4.3 | 6.5 | |
| Claude Opus 4.52026.04 | 95.5 | 1.13 | 4.3 | 6.5 | |
| Gemini 2.5 Flash2026.04 | 89.5 | 1.79 | 9.8 | 14.5 | |
| DeepSeek V3.12026.04 | 80.8 | 1.87 | 17 | 26 | |
| GPT 4.1 mini2026.04 | 72.7 | 3.6 | 26.3 | 36 | |
| GPT 5 mini2026.04 | 49 | 7.33 | 50 | 64 |