SQL execution performance on MMQA n=1105
50EM (1T)Oracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleSQL generator=Llama-3.2-3B-Instruct2026.01 | 50 | 42.9 | 42.9 | |
| DRR@5SQL generator=Gemma-3-4B-Instruct2026.01 | 50 | 13.2 | 13.3 | |
| OracleSQL generator=Gemma-3-4B-Instruct2026.01 | 50 | 38.1 | 38.1 | |
| DRR@5SQL generator=GPT-4o-mini2026.01 | 50 | 32 | 32 | |
| ReActSQL generator=GPT-4o-mini2026.01 | 50 | 28.8 | 28.9 | |
| CORE-TSQL generator=GPT-4o-mini2026.01 | 50 | 32.7 | 32.8 | |
| OracleSQL generator=GPT-4o-mini2026.01 | 50 | 63.4 | 63.3 | |
| DR@5SQL generator=Gemma-3-4B-Instruct2026.01 | 25 | 12.5 | 12.6 | |
| ReActSQL generator=Gemma-3-4B-Instruct2026.01 | 25 | 11.2 | 11.2 | |
| CORE-TSQL generator=Gemma-3-4B-Instruct2026.01 | 25 | 15.8 | 15.8 | |
| DR@5SQL generator=GPT-4o-mini2026.01 | 25 | 27.3 | 27.3 | |
| DR@5SQL generator=Llama-3.2-3B-Instruct2026.01 | 0 | 10.1 | 10 | |
| DRR@5SQL generator=Llama-3.2-3B-Instruct2026.01 | 0 | 11.4 | 11.3 | |
| ReActSQL generator=Llama-3.2-3B-Instruct2026.01 | 0 | 11.3 | 11.2 | |
| CORE-TSQL generator=Llama-3.2-3B-Instruct2026.01 | 0 | 15.6 | 15.6 |