Question Answering on WTQ (test)
78.38Accuracygpt-oss-20b
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| gpt-oss-20b2026.06 | 78.38 | 5.71 | 16.29 | 38.31 | |
| Qwen3-8BPrompting=plus prompting2026.06 | 77.51 | 12.5 | 28.76 | 48.25 | |
| Qwen3-8BTable Format=Markdown2026.06 | 77.26 | 14.34 | 32.69 | 48.15 | |
| Qwen3-8BPrompting=standard, Table Format=Default2026.06 | 77.14 | 14.04 | 32.63 | 46.89 | |
| Table-R1-Zero-7B2026.06 | 76.1 | 19.29 | 52.41 | 35.08 | |
| Qwen3-8BTable Format=CSV2026.06 | 75.94 | 17.54 | 37.32 | 48.82 | |
| Qwen3-4B2026.06 | 75.69 | 16.18 | 35.51 | 46.66 | |
| Distill-Llama-8B2026.06 | 59.78 | 37.96 | 60.45 | 35.96 | |
| Qwen3-1.7B2026.06 | 57.76 | 35.52 | 56.35 | 32.99 | |
| Qwen3-VL-8B-DISCOReasoning Strategy=Table-GLS, # TIR=02026.02 | 57.11 | — | — | — | |
| MiniCPM-V-2.6 8BReasoning Strategy=HIPPO, # TIR=55.2K2026.02 | 55.77 | — | — | — | |
| Llama4-Scout2026.06 | 55.71 | 46.48 | 72.77 | 30.66 | |
| Qwen2-VL-7B-Table-R1Reasoning Strategy=GRPO, # TIR=20.6K2026.02 | 50.3 | — | — | — | |
| Qwen3-VL-8B-DISCOReasoning Strategy=DA, # TIR=02026.02 | 50.16 | — | — | — | |
| Qwen3-VL-8BReasoning Strategy=DA, # TIR=02026.02 | 49.47 | — | — | — | |
| Distill-Qwen-7B2026.06 | 49.47 | 46.04 | 66.61 | 26.9 | |
| Qwen2.5-7B-Instruct2026.06 | 43.32 | 17.56 | 23.6 | 23.85 | |
| Gemma3n-E4B-DISCOReasoning Strategy=Table-GLS, # TIR=02026.02 | 41.32 | — | — | — | |
| Gemma3n-E4BReasoning Strategy=DA, # TIR=02026.02 | 30.5 | — | — | — | |
| GPT-4o-miniReasoning Strategy=DA, # TIR=02026.02 | 30.06 | — | — | — | |
| Gemma3n-E4B-DISCOReasoning Strategy=DA, # TIR=02026.02 | 24.56 | — | — | — | |
| Table-LLaVA 13BReasoning Strategy=SFT, # TIR=82K2026.02 | 20.41 | — | — | — | |
| Table-LLaVA 7BReasoning Strategy=SFT, # TIR=82K2026.02 | 18.43 | — | — | — |