Text-to-SQL on BIRD-147 2023 (test)
97.3SRRAPEX MAS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| APEX MASBase Model=GPT-4o, Time (s)=179, Cost ($)=0.362026.04 | 97.3 | 99.4 | 39 | 52.8 | 70.7 | |
| APEX MASBase Model=Sonnet, Time (s)=191, Cost ($)=0.452026.04 | 93.9 | 98.3 | 47 | 60.8 | 69.4 | |
| MAS CompilerTime (s)=64, Cost ($)=0.402026.04 | 79.6 | 93.6 | 94.8 | 93.7 | 67.4 | |
| Adaptive SkillTime (s)=55, Cost ($)=0.322026.04 | 77.6 | 92.7 | 95.3 | 93.4 | 68 | |
| Tools OnlyTime (s)=27, Cost ($)=0.232026.04 | 75.5 | 92.5 | 94.7 | 92.9 | 66 | |
| Knowledge OnlyTime (s)=12, Cost ($)=0.322026.04 | 74.2 | 92.5 | 94.9 | 93.1 | 66.7 | |
| Claude Code RawTime (s)=48, Cost ($)=0.332026.04 | 71.8 | 91.8 | 90 | 90.5 | 59.2 |