# Identifying Influential N-grams in Confidence Calibration via Regression Analysis Source: [Identifying Influential N-grams in Confidence Calibration via Regression Analysis]() Collection index: [llms.txt]() Data updated: 2026-04-09T05:00:04.534Z Authors: Shintaro Ozaki, Wataru Hashimoto, Hidetaka Kamigaito, Katsuhiko Hayashi, Taro Watanabe Published: 2026-04-07T11:58:49.000Z ## Abstract While large language models (LLMs) improve performance by explicit reasoning, their responses are often overconfident, even though they include linguistic expressions demonstrating uncertainty. In this work, we identify what linguistic expressions are related to confidence by applying the regression method. Specifically, we predict confidence of those linguistic expressions in the reasoning parts of LLMs as the dependent variables and analyze the relationship between a specific $n$-gram and confidence. Across multiple models and QA benchmarks, we show that LLMs remain overconfident when reasoning is involved and attribute this behavior to specific linguistic information. Interestingly, several of the extracted expressions coincide with cue phrases intentionally inserted on test-time scaling to improve reasoning performance. Through our test on causality and verification that the extracted linguistic information truly affects confidence, we reveal that confidence calibration is possible by simply suppressing those overconfident expressions without drops in performance. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Question Answering on MathQA, MMLU, HellaSwag, RACE, and CosmosQA](): Question Answering; MathQA, MMLU, HellaSwag, RACE, and CosmosQA; ECE: 0.079. ## Original sources and implementation [Original paper]() [PDF]()