Table Question Answering on WikiTableQuestions (WTQ) T (original tables) (sampled subset)
73.65AccuracyDP&PYAGENT w/ MIX-SC + NORM w/o RESORT
Evaluation results
| Method | Links | |
|---|---|---|
| DP&PYAGENT w/ MIX-SC + NORM w/o RESORTPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=DP&PyAgent, Self-consistency=Mix-SC, Normalization=NORM, NORM Reordering=false2023.12 | 73.65 | |
| DP&PYAGENT w/ MIX-SCPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=DP&PyAgent, Self-consistency=Mix-SC2023.12 | 73.06 | |
| DP&PYAGENT w/ MIX-SC + NORMPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=DP&PyAgent, Self-consistency=Mix-SC, Normalization=NORM2023.12 | 72.4 | |
| DATER w/ SCPrompting Protocol=Few-shot, Base Model=Codex, Self-consistency=true2023.12 | 68.99 | |
| DP w/ SC + NORM w/o RESORTPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=Direct Prompting, Self-consistency=true, Normalization=NORM, NORM Reordering=false2023.12 | 66.98 | |
| DP w/ SCPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=Direct Prompting, Self-consistency=true2023.12 | 66.39 | |
| NORM+DP&PYAGENT w/ EVALPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=DP&PyAgent, Normalization=NORM, Inference Evaluation=true2023.12 | 64.22 | |
| DP w/ SC + NORMPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=Direct Prompting, Self-consistency=true, Normalization=NORM2023.12 | 64.1 | |
| PYAGENT w/ SC + NORMPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=PyAgent, Self-consistency=true, Normalization=NORM2023.12 | 63.77 | |
| BINDERPrompting Protocol=Few-shot, Base Model=GPT-3.52023.12 | 63.61 | |
| PYAGENT w/ SC + NORM w/o RESORTPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=PyAgent, Self-consistency=true, Normalization=NORM, NORM Reordering=false2023.12 | 62.84 | |
| DATER w/o SCPrompting Protocol=Few-shot, Base Model=Codex, Self-consistency=false2023.12 | 61.75 | |
| PYAGENT w/ SCPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=PyAgent, Self-consistency=true2023.12 | 61.39 | |
| NORM+DPPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=Direct Prompting, Normalization=NORM2023.12 | 58.66 | |
| NORM+PYAGENTPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=PyAgent, Normalization=NORM2023.12 | 56.87 | |
| BINDERPrompting Protocol=Few-shot, Base Model=Codex2023.12 | 55.07 | |
| NORM+PYAGENT-OMITTEDPrompting Protocol=Zero-shot, Base Model=GPT-3.5, Reasoning Strategy=PyAgent-Omitted, Normalization=NORM2023.12 | 52.45 | |
| STRUCTGPTPrompting Protocol=Zero-shot, Base Model=GPT-3.52023.12 | 51.77 |