# ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models

Source: [ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models](<https://www.sota2.com/research/paper/68fa7c291e209131cea8cc49>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:58:13.123Z

Authors: Mingda Li, Xinyu Li, Weinan Zhang, Longxuan Ma

Published: 2025-10-15T02:46:43.000Z

## Abstract

Uncertainty Quantification (UQ) is a promising approach to improve model reliability, yet quantifying the uncertainty of Large Language Models (LLMs) is non-trivial. In this work, we establish a connection between the uncertainty of LLMs and their invariance under semantic-preserving intervention from a causal perspective. Building on this foundation, we propose a novel grey-box uncertainty quantification method that measures the variation in model outputs before and after the semantic-preserving intervention. Through theoretical justification, we show that our method provides an effective estimate of epistemic uncertainty. Our extensive experiments, conducted across various LLMs and a variety of question-answering (QA) datasets, demonstrate that our method excels not only in terms of effectiveness but also in computational efficiency.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.13103>)

[PDF](<https://arxiv.org/pdf/2510.13103.pdf>)
