# Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics

Source: [Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics](<https://www.sota2.com/research/paper/68fa76cd1e209131cea6f299>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T08:48:43.072Z

Authors: Ameya Godbole, Robin Jia

Published: 2025-01-24T19:17:06.000Z

## Abstract

Improvements in large language models have led to increasing optimism that they can serve as reliable evaluators of natural language generation outputs. In this paper, we challenge this optimism by thoroughly re-evaluating five state-of-the-art factuality metrics on a collection of 11 datasets for summarization, retrieval-augmented generation, and question answering. We find that these evaluators are inconsistent with each other and often misestimate system-level performance, both of which can lead to a variety of pitfalls. We further show that these metrics exhibit biases against highly paraphrased outputs and outputs that draw upon faraway parts of the source documents. We urge users of these factuality metrics to proceed with caution and manually validate the reliability of these metrics in their domain of interest before proceeding.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2501.14883>)

[PDF](<https://arxiv.org/pdf/2501.14883.pdf>)
