# Synthetic Contrastive Reasoning for Multi-Table Q&A

Source: [Synthetic Contrastive Reasoning for Multi-Table Q&A](<https://www.sota2.com/research/paper/6a22585886173ad9c14688f3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-06T05:00:04.316Z

Authors: Ankit Pratap Singh, Xin Su, Phillip Howard

Published: 2026-06-03T19:35:54.000Z

## Abstract

Multi-table question answering requires models to retrieve relevant evidence, link schemas, and perform compositional reasoning across relational tables. Existing multi-table Q&A resources typically provide questions and final answers but lack reasoning supervision that explains how answers are derived. To address this gap, we construct a synthetic contrastive reasoning-trace dataset for MMQA by generating validated positive traces and plausible negative traces with heterogeneous LLMs. We then use the resulting preference pairs to fine-tune open-weight LLMs with Contrastive Preference Optimization (CPO). Across Qwen3-14B, Mistral-8B, and Llama-3.1-8B, CPO achieves absolute average improvements over Q&A supervised fine-tuning ranging from 9.7%-16.3%, with gains up to 21 percentage points on MMQA. Ablations show that heterogeneous positive and negative trace generators strengthen the contrastive signal, and automated as well as human evaluations indicate that the generated pairs are largely faithful, coherent, and meaningfully contrastive.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Table Reasoning on MMQA (test)](<https://www.sota2.com/research/sota/table-reasoning-on-mmqa-test.md>): Table Reasoning; MMQA (test); Accuracy: 50.

- [Table Reasoning on MMTU (test)](<https://www.sota2.com/research/sota/table-reasoning-on-mmtu-test.md>): Table Reasoning; MMTU (test); Accuracy: 56.

- [Table Reasoning Accuracy on TableBench (test)](<https://www.sota2.com/research/sota/table-reasoning-accuracy-on-tablebench-test.md>): Table Reasoning; TableBench (test); Accuracy: 30.

- [Table Reasoning on BIRD (test)](<https://www.sota2.com/research/sota/table-reasoning-on-bird-test.md>): Table Reasoning; BIRD (test); Accuracy: 29.

- [Multi-table Reasoning on MMQA 3-table](<https://www.sota2.com/research/sota/multi-table-reasoning-on-mmqa-3-table.md>): Multi-table Reasoning; MMQA 3-table; Accuracy: 40.88.

- [Multi-table Reasoning on BIRD 3-table](<https://www.sota2.com/research/sota/multi-table-reasoning-on-bird-3-table.md>): Multi-table Reasoning; BIRD 3-table; Accuracy: 56.25.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.05382>)

[PDF](<https://arxiv.org/pdf/2606.05382.pdf>)
