Excel Formula Generation on WikiTableQuestions (WikiTQ) 639 solvable problems (test)
84.19Execution Match (EM)GPT-4o
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4oAugmentation=RAGOracle2025.03 | 84.19 | |
| GPT-4oAugmentation=Base Model2025.03 | 79.19 | |
| GPT-4oAugmentation=RAGAll2025.03 | 78.25 | |
| Qwen2.5-coder 14BAugmentation=FTSyn-QA2025.03 | 54.93 | |
| Qwen2.5-coder 14BAugmentation=RAGOracle2025.03 | 53.52 | |
| Qwen2.5 14BAugmentation=FTSyn-QA2025.03 | 51.02 | |
| Qwen2.5 14BAugmentation=Base Model2025.03 | 50.7 | |
| Qwen2.5 14BAugmentation=FTDoc-QA2025.03 | 49.92 | |
| Qwen2.5 14BAugmentation=RAGOracle2025.03 | 49.45 | |
| Qwen2.5 14BAugmentation=RAGAll2025.03 | 47.57 | |
| Qwen2.5-coder 14BAugmentation=Base Model2025.03 | 46.95 | |
| Qwen2.5-coder 14BAugmentation=FTDoc-QA2025.03 | 46.95 | |
| Qwen2.5 14BAugmentation=FTDoc2025.03 | 46.64 | |
| Qwen2.5-coder 14BAugmentation=RAGAll2025.03 | 44.76 | |
| Qwen2.5-coder 14BAugmentation=FTDoc2025.03 | 40.22 | |
| Qwen2.5-coder 3BAugmentation=FTSyn-QA2025.03 | 28.64 | |
| CodeLlama-Instruct 13BAugmentation=FTSyn-QA2025.03 | 21.28 | |
| Qwen2.5 3BAugmentation=FTSyn-QA2025.03 | 20.81 | |
| Qwen2.5-coder 3BAugmentation=RAGOracle2025.03 | 18.94 | |
| Qwen2.5 3BAugmentation=RAGOracle2025.03 | 17.84 | |
| CodeLlama-Instruct 13BAugmentation=RAGOracle2025.03 | 16.12 | |
| Llama2 13BAugmentation=FTSyn-QA2025.03 | 15.49 | |
| Qwen2.5-coder 3BAugmentation=Base Model2025.03 | 15.34 | |
| Qwen2.5-coder 3BAugmentation=FTDoc-QA2025.03 | 15.34 | |
| Qwen2.5 3BAugmentation=Base Model2025.03 | 14.87 | |
| Qwen2.5 3BAugmentation=FTDoc-QA2025.03 | 14.55 | |
| Qwen2.5-coder 3BAugmentation=RAGAll2025.03 | 13.77 | |
| Qwen2.5-coder 3BAugmentation=FTDoc2025.03 | 13.62 | |
| Qwen2.5 3BAugmentation=FTDoc2025.03 | 13.62 | |
| CodeLlama-Instruct 13BAugmentation=RAGAll2025.03 | 13.15 | |
| CodeLlama-Instruct 13BAugmentation=FTDoc-QA2025.03 | 12.83 | |
| Qwen2.5 3BAugmentation=RAGAll2025.03 | 12.21 | |
| CodeLlama-Instruct 13BAugmentation=FTDoc2025.03 | 12.21 | |
| CodeLlama-Instruct 13BAugmentation=Base Model2025.03 | 10.8 | |
| CodeLlama-Instruct 7BAugmentation=FTDoc-QA2025.03 | 9.23 | |
| Llama2 7BAugmentation=FTSyn-QA2025.03 | 8.61 | |
| CodeLlama-Instruct 7BAugmentation=FTSyn-QA2025.03 | 4.85 | |
| Llama2 13BAugmentation=FTDoc-QA2025.03 | 4.54 | |
| Llama2 7BAugmentation=FTDoc-QA2025.03 | 3.91 | |
| Llama2 13BAugmentation=Base Model2025.03 | 1.72 | |
| Llama2 13BAugmentation=RAGOracle2025.03 | 1.56 | |
| Llama2 7BAugmentation=RAGOracle2025.03 | 1.1 | |
| CodeLlama-Instruct 7BAugmentation=FTDoc2025.03 | 0.78 | |
| Llama2 7BAugmentation=Base Model2025.03 | 0.63 | |
| Llama2 7BAugmentation=FTDoc2025.03 | 0.63 | |
| CodeLlama-Instruct 7BAugmentation=Base Model2025.03 | 0.47 | |
| CodeLlama-Instruct 7BAugmentation=RAGOracle2025.03 | 0.47 | |
| Llama2 13BAugmentation=FTDoc2025.03 | 0.47 | |
| CodeLlama-Instruct 7BAugmentation=RAGAll2025.03 | 0.31 | |
| Llama2 13BAugmentation=RAGAll2025.03 | 0.31 | |
| Llama2 7BAugmentation=RAGAll2025.03 | 0.16 |