Text-to-SQL on Spider 2.0
4.4Llama-8B Execution AccuracyOracle
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| OracleRetrieval category=Oracle2026.04 | 4.4 | 7.2 | 3.5 | 7.4 | 1.4 | 7.6 | |
| ATRRetrieval category=Ours, Retrieval backbone=UAE-Large-V12026.04 | 1.6 | 4.5 | 2.1 | 4.9 | 1.1 | 4.6 | |
| UAE-Large-V1k=10, Retrieval category=Encoder-based, Retrieval backbone=UAE-Large-V12026.04 | 1.4 | 4.1 | 1.4 | 3.4 | 0.5 | 3.9 | |
| Contrieverk=3, Retrieval category=Encoder-based, Retrieval backbone=Contriever2026.04 | 1.1 | 3.4 | 0.5 | 3.9 | 0.2 | 2.5 | |
| Contrieverk=5, Retrieval category=Encoder-based, Retrieval backbone=Contriever2026.04 | 1.1 | 3.2 | 1.4 | 3.7 | 0.7 | 3.2 | |
| Contrieverk=10, Retrieval category=Encoder-based, Retrieval backbone=Contriever2026.04 | 1.1 | 3.5 | 0.9 | 4.1 | 0.2 | 3.4 | |
| RankZephyrk=3, Retrieval category=LLM-based2026.04 | 1.1 | 3.2 | 1.4 | 4.1 | 0.2 | 2.5 | |
| RankZephyrk=10, Retrieval category=LLM-based2026.04 | 1.1 | 3.4 | 1.4 | 3.4 | 0.5 | 4.4 | |
| JARk=5, Retrieval category=Reranker-based, Retrieval backbone=Contriever2026.04 | 1.1 | 3.9 | 0.7 | 3.7 | 0.9 | 3.2 | |
| ATRRetrieval category=Ours, Retrieval backbone=Contriever2026.04 | 1.1 | 4.4 | 1.4 | 5.7 | 1.1 | 4.8 | |
| UAE-Large-V1k=5, Retrieval category=Encoder-based, Retrieval backbone=UAE-Large-V12026.04 | 0.9 | 3.9 | 1.1 | 3.4 | 0.9 | 4.1 | |
| RankZephyrk=5, Retrieval category=LLM-based2026.04 | 0.9 | 3 | 1.1 | 4.1 | 0.7 | 3 | |
| Murrek=3, Retrieval category=LLM-based2026.04 | 0.9 | 4.1 | 1.4 | 3.2 | 0.9 | 4.1 | |
| Murrek=5, Retrieval category=LLM-based2026.04 | 0.9 | 3 | 1.4 | 3.7 | 0.9 | 3 | |
| JARk=10, Retrieval category=Reranker-based, Retrieval backbone=Contriever2026.04 | 0.9 | 3 | 1.4 | 4.4 | 0.5 | 4.1 | |
| JARk=3, Retrieval category=Reranker-based, Retrieval backbone=UAE-Large-V12026.04 | 0.9 | 3.9 | 1.6 | 4.4 | 0.5 | 2.8 | |
| JARk=5, Retrieval category=Reranker-based, Retrieval backbone=UAE-Large-V12026.04 | 0.9 | 4.1 | 1.4 | 4.1 | 0.9 | 4.4 | |
| JARk=10, Retrieval category=Reranker-based, Retrieval backbone=UAE-Large-V12026.04 | 0.9 | 3 | 1.6 | 3.7 | 0.9 | 3.2 | |
| UAE-Large-V1k=3, Retrieval category=Encoder-based, Retrieval backbone=UAE-Large-V12026.04 | 0.7 | 3.5 | 1.1 | 3.4 | 0.5 | 3.4 | |
| Murrek=10, Retrieval category=LLM-based2026.04 | 0.7 | 4.1 | 1.4 | 4.4 | 0.7 | 4.6 | |
| JARk=3, Retrieval category=Reranker-based, Retrieval backbone=Contriever2026.04 | 0.7 | 3.7 | 0.9 | 4.1 | 0.5 | 3 |