SQL Execution Performance on Spider n=1034
84.3EM (1 Table)Oracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleSQL generator=GPT-4o-mini2026.01 | 84.3 | 63 | 74.9 | |
| OracleSQL generator=Gemma-3-4B-Instruct2026.01 | 77 | 41.4 | 61.2 | |
| DR@5SQL generator=GPT-4o-mini2026.01 | 73.7 | 53.8 | 64.9 | |
| ReActSQL generator=GPT-4o-mini2026.01 | 73.2 | 51.6 | 63.6 | |
| DRR@5SQL generator=GPT-4o-mini2026.01 | 72.3 | 58 | 66 | |
| CORE-TSQL generator=GPT-4o-mini2026.01 | 71.5 | 55.8 | 64.5 | |
| JAR@5SQL generator=GPT-4o-mini2026.01 | 71.3 | 53.2 | 63.2 | |
| OracleSQL generator=Llama-3.2-3B-Instruct2026.01 | 60.9 | 46.6 | 54.5 | |
| DR@5SQL generator=Gemma-3-4B-Instruct2026.01 | 60.9 | 30.7 | 47.5 | |
| DRR@5SQL generator=Gemma-3-4B-Instruct2026.01 | 59.8 | 29.2 | 46.2 | |
| JAR@5SQL generator=Gemma-3-4B-Instruct2026.01 | 59.3 | 32.2 | 47.3 | |
| CORE-TSQL generator=Gemma-3-4B-Instruct2026.01 | 58.6 | 35.7 | 48.5 | |
| ReActSQL generator=Gemma-3-4B-Instruct2026.01 | 58.4 | 30.3 | 45.9 | |
| ReActSQL generator=Llama-3.2-3B-Instruct2026.01 | 35.8 | 25.1 | 31 | |
| CORE-TSQL generator=Llama-3.2-3B-Instruct2026.01 | 35.8 | 30.1 | 33.3 | |
| JAR@5SQL generator=Llama-3.2-3B-Instruct2026.01 | 32.7 | 25.5 | 29.5 | |
| DR@5SQL generator=Llama-3.2-3B-Instruct2026.01 | 31.7 | 24 | 28.2 | |
| DRR@5SQL generator=Llama-3.2-3B-Instruct2026.01 | 31.1 | 24 | 27.9 |