Table Question Answering on SSTQA (test)
78.14EMASTRA (Oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| ASTRA (Oracle)Backbone=Qwen3-8B, Thinking Mode=thinking2026.04 | 78.14 | |
| ASTRA (Oracle)Backbone=Qwen3-8B, Thinking Mode=no thinking2026.04 | 75.26 | |
| ASTRA (Adaptive Selection)Backbone=Qwen3-8B, Thinking Mode=thinking2026.04 | 70.02 | |
| ASTRA (Textual Reasoning)Backbone=Qwen3-8B, Thinking Mode=thinking2026.04 | 65.45 | |
| ASTRA (Adaptive Selection)Backbone=Qwen3-8B, Thinking Mode=no thinking2026.04 | 65.31 | |
| Direct Prompt (Semantic Tree)Backbone=Qwen3-8B, Thinking Mode=thinking2026.04 | 64.92 | |
| ASTRA (Symbolic Reasoning)Backbone=Qwen3-8B, Thinking Mode=thinking2026.04 | 62.7 | |
| Direct Prompt (Semantic Tree)Backbone=Qwen3-8B, Thinking Mode=no thinking2026.04 | 60.86 | |
| ASTRA (Oracle)Backbone=Qwen2.5-7B2026.04 | 60.64 | |
| ASTRA (Textual Reasoning)Backbone=Qwen3-8B, Thinking Mode=no thinking2026.04 | 60.6 | |
| ASTRA (Symbolic Reasoning)Backbone=Qwen3-8B, Thinking Mode=no thinking2026.04 | 58.64 | |
| Direct Prompt (Raw Table)Backbone=Qwen3-8B, Thinking Mode=thinking2026.04 | 54.97 | |
| ASTRA (Adaptive Selection)Backbone=Qwen2.5-7B2026.04 | 53.01 | |
| Direct Prompt (Raw Table)Backbone=Qwen3-8B, Thinking Mode=no thinking2026.04 | 52.75 | |
| ASTRA (Textual Reasoning)Backbone=Qwen2.5-7B2026.04 | 50.14 | |
| Direct Prompt (Semantic Tree)Backbone=Qwen2.5-7B2026.04 | 49.74 | |
| Direct Prompt (Raw Table)Backbone=Qwen2.5-7B2026.04 | 46.6 | |
| ASTRA (Oracle)Backbone=Llama3.1-8B2026.04 | 44.76 | |
| ASTRA (Adaptive Selection)Backbone=Llama3.1-8B2026.04 | 41.49 | |
| ASTRA (Textual Reasoning)Backbone=Llama3.1-8B2026.04 | 40.45 | |
| Direct Prompt (Semantic Tree)Backbone=Llama3.1-8B2026.04 | 37.43 | |
| ASTRA (Symbolic Reasoning)Backbone=Qwen2.5-7B2026.04 | 36.39 | |
| Direct Prompt (Raw Table)Backbone=Llama3.1-8B2026.04 | 35.99 | |
| ASTRA (Symbolic Reasoning)Backbone=Llama3.1-8B2026.04 | 10.47 |