# FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining

Source: [FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining](<https://www.sota2.com/research/paper/68fa6c1a1e209131cea32324>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:07:39.515Z

Authors: Zhoujun Cheng, Haoyu Dong, Ran Jia, Pengfei Wu, Shi Han, Fan Cheng, Dongmei Zhang

Published: 2021-09-15T14:31:17.000Z

## Abstract

Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, which performs calculations on numerical values in tables, is naturally a strong supervision of numerical reasoning. More importantly, large amounts of spreadsheets with expert-made formulae are available on the web and can be obtained easily. FORTAP is the first method for numerical-reasoning-aware table pretraining by leveraging large corpus of spreadsheet formulae. We design two formula pretraining tasks to explicitly guide FORTAP to learn numerical reference and calculation in semi-structured tables. FORTAP achieves state-of-the-art results on two representative downstream tasks, cell type classification and formula prediction, showing great potential of numerical-reasoning-aware pretraining.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Formula Prediction on Enron (test)](<https://www.sota2.com/research/sota/formula-prediction-on-enron-test.md>): Formula Prediction; Enron (test); Formula Score: 55.8.

- [Table Question Answering on HiTab (test)](<https://www.sota2.com/research/sota/table-question-answering-on-hitab-test.md>): Table Question Answering; HiTab (test); Execution Accuracy: 47.

- [Cell Type Classification on DeEx (test)](<https://www.sota2.com/research/sota/cell-type-classification-on-deex-test.md>): Cell Type Classification; DeEx (test); Macro Accuracy: 85.2.

- [Table Question Answering on HiTab (dev) using Execution Accuracy](<https://www.sota2.com/research/sota/table-question-answering-on-hitab-dev-using-execution-accuracy.md>): Table Question Answering; HiTab (dev); Execution Accuracy: 47.1.

- [Hierarchical Table Question Answering on HiTab](<https://www.sota2.com/research/sota/hierarchical-table-question-answering-on-hitab.md>): Hierarchical Table Question Answering; HiTab; Accuracy: 47.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2109.07323>)

[PDF](<https://arxiv.org/pdf/2109.07323.pdf>)

[Implementation](<https://github.com/microsoft/TUTA_table_understanding>)