Text-to-SQL on BIRD (MAT, Token/s)
6.67Match Accuracy (MAT)HedgeSpec
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HedgeSpecTarget Model=Llama-3.1-8B2025.10 | 6.67 | 83.87 | |
| EAGLETarget Model=Llama-3.1-8B2025.10 | 5.5 | 73.24 | |
| HedgeSpecTarget Model=Qwen-3-8B2025.10 | 5.18 | 56.12 | |
| HedgeSpecTarget Model=Qwen-3-32B2025.10 | 4.92 | 34.72 | |
| UCBSpecTarget Model=Llama-3.1-8B2025.10 | 4.67 | 62.6 | |
| EXP3SpecTarget Model=Llama-3.1-8B2025.10 | 4.38 | 58.03 | |
| EAGLETarget Model=Qwen-3-8B2025.10 | 4.08 | 47.91 | |
| UCBSpecTarget Model=Qwen-3-8B2025.10 | 3.94 | 45.48 | |
| UCBSpecTarget Model=Qwen-3-32B2025.10 | 3.85 | 27.56 | |
| EXP3SpecTarget Model=Qwen-3-32B2025.10 | 3.78 | 27.36 | |
| EXP3SpecTarget Model=Qwen-3-8B2025.10 | 3.73 | 43.67 | |
| EAGLETarget Model=Qwen-3-32B2025.10 | 2.95 | 21.5 | |
| Llama-3.1-8BTarget Model=Llama-3.1-8B2025.10 | 1 | 17.95 | |
| Qwen-3-8BTarget Model=Qwen-3-8B2025.10 | 1 | 14.91 | |
| Qwen-3-32BTarget Model=Qwen-3-32B2025.10 | 1 | 8.43 |