# Core: Robust Factual Precision with Informative Sub-Claim Identification

Source: [Core: Robust Factual Precision with Informative Sub-Claim Identification](<https://www.sota2.com/research/paper/68fa753e1e209131cea66b85>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T07:29:39.376Z

Authors: Zhengping Jiang, Jingyu Zhang, Nathaniel Weir, Seth Ebner, Miriam Wanner, Kate Sanders, Daniel Khashabi, Anqi Liu, Benjamin Van Durme

Published: 2024-07-04T01:51:38.000Z

## Abstract

Hallucinations pose a challenge to the application of large language models (LLMs) thereby motivating the development of metrics to evaluate factual precision. We observe that popular metrics using the Decompose-Then-Verify framework, such as \\FActScore, can be manipulated by adding obvious or repetitive subclaims to artificially inflate scores. This observation motivates our new customizable plug-and-play subclaim selection component called Core, which filters down individual subclaims according to their uniqueness and informativeness. We show that many popular factual precision metrics augmented by Core are substantially more robust on a wide range of knowledge domains. We release an evaluation framework supporting easy and modular use of Core and various decomposition strategies, which we recommend adoption by the community. We also release an expansion of the FActScore biography dataset to facilitate further studies of decomposition-based factual precision evaluation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Factual Precision Evaluation on Bios](<https://www.sota2.com/research/sota/factual-precision-evaluation-on-bios.md>): Factual Precision Evaluation; BIOS; FACTSCORE: 83.

- [Factual Precision Evaluation on Culture & Entertainment](<https://www.sota2.com/research/sota/factual-precision-evaluation-on-culture-and-entertainment.md>): Factual Precision Evaluation; Culture & Entertainment; FACTSCORE: 0.887.

- [Factual Precision Evaluation on Geographic](<https://www.sota2.com/research/sota/factual-precision-evaluation-on-geographic.md>): Factual Precision Evaluation; Geographic; FactScore: 86.7.

- [Factual Precision Evaluation on Medical](<https://www.sota2.com/research/sota/factual-precision-evaluation-on-medical.md>): Factual Precision Evaluation; medical; SAFE: 87.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2407.03572>)

[PDF](<https://arxiv.org/pdf/2407.03572.pdf>)

[Implementation](<https://github.com/zipJiang/adversarial-factuality>)
