Text-to-SQL on KaggleDBQA (test)
64EA (%)MNL
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| MNLBase Model=DeepSeek-V3.2-Exp2025.12 | 64 | 54 | 514 | |
| ACEBase Model=DeepSeek-V3.2-Exp2025.12 | 54 | 96 | 9,406 | |
| TFGOBase Model=Qwen3-Max2025.12 | 47 | — | 125 | |
| MementoBase Model=Qwen3-Max2025.12 | 47 | 87 | 992 | |
| MNLBase Model=Qwen3-Max2025.12 | 46 | 54 | 375 | |
| VanillaBase Model=Qwen3-Max2025.12 | 40 | — | — | |
| MNLBase Model=Qwen3-8B2025.12 | 28 | 50 | 752 | |
| VanillaBase Model=DeepSeek-V3.2-Exp2025.12 | 24 | — | — | |
| TFGOBase Model=DeepSeek-V3.2-Exp2025.12 | 24 | — | 100 | |
| TFGOBase Model=Qwen3-8B2025.12 | 22 | — | 34 | |
| ACEBase Model=Qwen3-8B2025.12 | 22 | 98 | 6,289 | |
| VanillaBase Model=Qwen3-8B2025.12 | 19 | — | — | |
| MementoBase Model=DeepSeek-V3.2-Exp2025.12 | 19 | 87 | 1,419 | |
| MementoBase Model=Qwen3-8B2025.12 | 15 | 87 | 530 |