List your company
ResearchFormerly WizWand

Table Fact Verification on TABFACT small (test)

0.921AccuracyHuman Performance
0.487320.599910.71250.82509Oct 1, 2020May 14, 2021Dec 25, 2021Aug 8, 2022Mar 21, 2023Nov 1, 2023Jun 14, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Human Performance2020.100.921——
Human Performance2021.070.921——
Human Performance2022.110.921——
PASTA2022.110.906——
SaMoE2022.110.867——
DeBERTaV32022.110.865——
REASTAP2022.100.862——
TAPEXpre-training_steps=50000, fine-tuning_steps=200002021.070.859——
Tapex2022.110.859——
Tab-PoTLLM=DeepSeek-67B2024.060.857791.3480.27
DaterLLM=Codex2024.060.85691.280
TAPEX2022.100.855——
BinderLLM=Not Specified2024.060.851——
DecompTaPas2022.100.847——
TableFormer2022.100.846——
TAPASModel=Large, Pre-training Objective=Counterfactual + Synthetic2020.100.839——
Eisenschlos et al.model_category=Pre-trained Language Models2021.070.839——
TaPas2022.100.839——
Tapas2022.110.839——
COTLLM=DeepSeek-67B2024.060.826188.4676.84
BART2021.070.825——
BART2022.100.823——
Tab-PoTLLM=Mixtral-8x7B2024.060.816788.3675.07
TAPASModel=Base, Pre-training Objective=Counterfactual + Synthetic2020.100.81——
TAPASModel=Base, Pre-training Objective=Synthetic2020.100.804——
COTLLM=Mixtral-8x7B2024.060.800986.0774.19
Schlichtkrull et al.Retrieval=Oracle2022.110.794——
DirectLLM=DeepSeek-67B2024.060.786184.6872.62
TAPASModel=Base, Pre-training Objective=Counterfactual2020.100.774——
TAPASModel=Base, Pre-training Objective=SQA2020.100.773——
Yang et al.model_category=Pre-trained Language Models2021.070.762——
ProgVGAT2022.110.762——
DirectLLM=Mixtral-8x7B2024.060.758981.2970.56
POTLLM=DeepSeek-67B2024.060.745176.3272.72
LOGICALFACTCHECKER (program from Seq2Action)2020.100.743——
Zhong et al.model_category=Pre-trained Language Models2021.070.743——
LFC2022.100.743——
LogicFactChecker2022.110.743——
LOGICALFACTCHECKER (program from LPA)2020.100.742——
Shi et al.model_category=Pre-trained Language Models2021.070.742——
HeterTFV2022.100.742——
POTLLM=Mixtral-8x7B2024.060.733276.0270.66
TAPASModel=Base, Pre-training Objective=MASK-LM2020.100.722——
Tab-PoTLLM=Mixtral-7B2024.060.718476.8266.93
COTLLM=Mixtral-7B2024.060.704173.7367.12
LPA-Ranking w/ Discriminator (Caption)2020.100.689——
LPA-Ranking2022.100.689——
TABLE-BERT-Horizontal-T+F-Template2020.100.681——
Chen et al.model_category=Pre-trained Language Models2021.070.681——
Table-BERT2022.110.681——
DirectLLM=Mixtral-7B2024.060.678973.1362.71
POTLLM=Mixtral-7B2024.060.662565.9766.54
Tab-PoTLLM=DeepSeek-7B2024.060.658670.7561.04
COTLLM=DeepSeek-7B2024.060.651770.6559.76
POTLLM=DeepSeek-7B2024.060.619164.8858.98
DirectLLM=DeepSeek-7B2024.060.577159.555.94
BERT classifier w/o Table2020.100.504——