# Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales

Source: [Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales](<https://www.sota2.com/research/paper/6a1e6403dbd7150506f855c7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T05:02:08.515Z

Authors: Pingjun Hong, Benjamin Roth

Published: 2026-05-31T10:35:35.000Z

## Abstract

Natural-language explanations are often treated as a unified interface for understanding model behavior, but different explanation sources may support simulation in different ways. This paper compares two families of explanations for question answering models: verbalized feature attributions and self-generated rationales. We evaluate them under a shared counterfactual simulation setting, using an LLM judge as predictor and measuring whether it can better predict a model's answers to follow-up questions when given its explanation. Across multiple instruction-tuned models, we analyze how explanation source, verbalization strategy, and feature granularity affect the simulatability of explanations. Our results show that explanation format and granularity affect simulatability: attribution-based explanations and self-generated rationales differ in how much they improve counterfactual prediction, with effects that vary across models and formats.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.01148>)

[PDF](<https://arxiv.org/pdf/2606.01148.pdf>)
