NL2SQL on BIRD
78.7California Schools AccuracyCustom Orchestrator (V5)
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Custom Orchestrator (V5)Scenario=V5, Features=+Hybrid Models2026.05 | 78.7 | 73.7 | 65.6 | 75.8 | 76.4 | 71.8 | 86.7 | 95.3 | 84.1 | 62.6 | 69.6 | 76.3 | |
| Custom Orchestrator (V6)Scenario=V6, Features=+Struct. Context2026.05 | 75.3 | 76.8 | 71.9 | 79.1 | 79.2 | 70.7 | 91.1 | 96.1 | 84.1 | 63.2 | 73.3 | 78.1 | |
| Custom Orchestrator (V4)Scenario=V4, Features=+Vector Search2026.05 | 74.2 | 73.1 | 64.2 | 78.3 | 75.5 | 70.1 | 88 | 94.6 | 83.4 | 62.6 | 71.2 | 76 | |
| Autogen Orchestrator (V3)Scenario=V3, Features=+Hybrid Models2026.05 | 73 | 70.4 | 79.6 | 80.6 | 76.4 | 71.2 | 87.9 | 89.9 | 81.4 | 60.7 | 67.5 | 75.4 | |
| Autogen Orchestrator (V2)Scenario=V2, Features=+Vector Search2026.05 | 67.4 | 63.3 | 76.5 | 75 | 70.8 | 69.4 | 83.5 | 89.2 | 81.4 | 64.4 | 64.5 | 72.5 | |
| Autogen Orchestrator (V1)Scenario=V1, Features=No Vec. Search2026.05 | 55.7 | 61.1 | 59.4 | 72.9 | 66.7 | 69.5 | 83.5 | 88.4 | 67.6 | 60.1 | 64.2 | 68.5 |