# TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness

Source: [TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness](<https://www.sota2.com/research/paper/68fa73281e209131cea5ac7a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:43:27.234Z

Authors: Danna Zheng, Danyang Liu, Mirella Lapata, Jeff Z. Pan

Published: 2024-02-19T21:12:14.000Z

## Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, prompting a surge in their practical applications. However, concerns have arisen regarding the trustworthiness of LLMs outputs, particularly in closed-book question-answering tasks, where non-experts may struggle to identify inaccuracies due to the absence of contextual or ground truth information. This paper introduces TrustScore, a framework based on the concept of Behavioral Consistency, which evaluates whether an LLMs response aligns with its intrinsic knowledge. Additionally, TrustScore can seamlessly integrate with fact-checking methods, which assesses alignment with external knowledge sources. The experimental results show that TrustScore achieves strong correlations with human judgments, surpassing existing reference-free metrics, and achieving results on par with reference-based metrics.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2402.12545>)

[PDF](<https://arxiv.org/pdf/2402.12545.pdf>)
