List your company
ResearchFormerly WizWand

NL-to-SQL on WikiTQ

59.7Execution AccuracyTable-Specialist
-2.2444813.8372629.91946.00074Oct 16, 2024Jan 22, 2025Apr 30, 2025Aug 6, 2025Nov 12, 2025Feb 18, 2026May 27, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.1059.7
GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.1055.9
Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.1051.3
Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.1044.9
GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.1042.1
Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.1030
Table-SpecialistBase Model=GPT-4, Training Strategy=Table-Specialist2024.100.55
ReflectionInference Method=Best-of-N (pass@5), Temperature=0.92026.050.49
ReflectionInference Method=MCTS2026.050.449
GPT-4Base Model=GPT-4, Training Strategy=Vanilla2024.100.416
No MemoryInference Method=Best-of-N (pass@5), Temperature=0.92026.050.408
Table-SpecialistBase Model=GPT-3.5, Training Strategy=Table-Specialist2024.100.404
No MemoryInference Method=MCTS2026.050.388
LiTS-FactInference Method=MCTS2026.050.388
Raw SiblingInference Method=MCTS2026.050.367
LiTS-FactInference Method=Best-of-N (pass@5), Temperature=0.92026.050.347
Raw SiblingInference Method=Beam Search2026.050.306
ReActInference Method=Single Trajectory, Temperature=02026.050.286
No MemoryInference Method=Beam Search2026.050.265
Table-SpecialistBase Model=Llama3.1-8B, Training Strategy=Table-Specialist2024.100.257
GPT-3.5Base Model=GPT-3.5, Training Strategy=Vanilla2024.100.209
Llama3.1-8BBase Model=Llama3.1-8B, Training Strategy=Vanilla2024.100.138