Natural Language to SQL on WikiSQL (test)
88.7AccuracyX-SQL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| X-SQLSupervision setting=Fully-supervised2019.09 | 88.7 | — | — | |
| ShapLoRABackbone=LlaMA-3 8B, Number of random seeds=52026.01 | 88.5 | — | — | |
| AutoLoRABackbone=LlaMA-3 8B, Number of random seeds=52026.01 | 87.5 | — | — | |
| BitFitBackbone=LlaMA-3 8B, Number of random seeds=52026.01 | 87.3 | — | — | |
| LoRABackbone=LlaMA-3 8B, Number of random seeds=52026.01 | 87.3 | — | — | |
| AdaLoRABackbone=LlaMA-3 8B, Number of random seeds=52026.01 | 86.9 | — | — | |
| Housbly-AdapterBackbone=LlaMA-3 8B, Number of random seeds=52026.01 | 86.8 | — | — | |
| P-tuning v2Backbone=LlaMA-3 8B, Number of random seeds=52026.01 | 86.4 | — | — | |
| SQLovaSupervision setting=Fully-supervised2019.09 | 86.2 | — | — | |
| Discrete Hard EMSupervision setting=Weakly-supervised2019.09 | 83.9 | — | — | |
| Coarse2FineSupervision setting=Fully-supervised2019.09 | 78.5 | — | — | |
| MAPOX+MERLSupervision setting=Weakly-supervised2019.09 | 74.8 | — | — | |
| MAPOXSupervision setting=Weakly-supervised2019.09 | 74.2 | — | — | |
| TypeSQLSupervision setting=Fully-supervised2019.09 | 73.5 | — | — | |
| MAPOSupervision setting=Weakly-supervised2019.09 | 72.4 | — | — | |
| MMLSupervision setting=Weakly-supervised2019.09 | 70.5 | — | — | |
| SQLNetSupervision setting=Fully-supervised2019.09 | 68 | — | — | |
| Max lossYear=2017, Meta-learning=false2018.03 | — | 60.5 | 65.8 | |
| Meta + Max lossLearner=Max loss, Meta-learning=true2018.03 | — | 61.6 | 67 | |
| Meta + Pointer lossLearner=Pointer loss, Meta-learning=true2018.03 | — | 51.4 | 57.2 | |
| Meta + Sum lossLearner=Sum loss, Meta-learning=true2018.03 | — | 62.8 | 68 | |
| Pointer lossYear=2017, Meta-learning=false2018.03 | — | 46.1 | 51.8 | |
| PointerNetYear=20172018.03 | — | 43.3 | 53.3 | |
| Seq2SQLYear=20172018.03 | — | 48.3 | 59.4 | |
| Sum lossYear=2017, Meta-learning=false2018.03 | — | 61.5 | 66.8 |