# IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration

Source: [IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration](<https://www.sota2.com/research/paper/69df1bb57d0172140ec7ba5e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-16T05:00:11.996Z

Authors: Yanji He, Yuxin Jiang, Yiwen Wu, Bo Huang, Jiaheng Wei, Wei Wang

Published: 2026-04-14T10:50:49.000Z

## Abstract

Large Language Models are increasingly deployed for decision-making, yet their adoption in high-stakes domains remains limited by miscalibrated probabilities, unfaithful explanations, and inability to incorporate expert knowledge precisely. We propose IDEA, a framework that extracts LLM decision knowledge into an interpretable parametric model over semantically meaningful factors. Through joint learning of verbal-to-numerical mappings and decision parameters via EM, correlated sampling that preserves factor dependencies, and direct parameter editing with mathematical guarantees, IDEA produces calibrated probabilities while enabling quantitative human-AI collaboration. Experiments across five datasets show IDEA with Qwen-3-32B (78.6%) outperforms DeepSeek R1 (68.1%) and GPT-5.2 (77.9%), achieving perfect factor exclusion and exact calibration -- precision unattainable through prompting alone. The implementation is publicly available at https://github.com/leonbig/IDEA.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Three-way probability ranking on COMMON2SENSE paired](<https://www.sota2.com/research/sota/three-way-probability-ranking-on-common2sense-paired.md>): Three-way probability ranking; COMMON2SENSE paired; F1 (C1): 76.2.

- [Binary decision on BIGDATA 22](<https://www.sota2.com/research/sota/binary-decision-on-bigdata-22.md>): Binary decision; BIGDATA 22; Accuracy: 69.3.

- [Binary Classification on German Credit (Accuracy)](<https://www.sota2.com/research/sota/binary-classification-on-german-credit-accuracy.md>): Binary decision; German Credit; Accuracy: 68.9.

- [Binary decision on COMMON2SENSE](<https://www.sota2.com/research/sota/binary-decision-on-common2sense.md>): Binary decision; COMMON2SENSE; Accuracy: 95.1.

- [Binary decision on PLASMA](<https://www.sota2.com/research/sota/binary-decision-on-plasma.md>): Binary decision; PLASMA; Accuracy: 84.5.

- [Binary decision on TODAY](<https://www.sota2.com/research/sota/binary-decision-on-today.md>): Binary decision; TODAY; Accuracy: 75.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.12573>)

[PDF](<https://arxiv.org/pdf/2604.12573.pdf>)