# Reasoning Elicitation in Language Models via Counterfactual Feedback

Source: [Reasoning Elicitation in Language Models via Counterfactual Feedback](<https://www.sota2.com/research/paper/68fa777b1e209131cea737cf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:19:49.174Z

Authors: Alihan H\\"uy\\"uk, Xinnuo Xu, Jacqueline Maasch, Aditya V. Nori, Javier Gonz\\'alez

Published: 2024-10-02T15:33:30.000Z

## Abstract

Despite the increasing effectiveness of language models, their reasoning capabilities remain underdeveloped. In particular, causal reasoning through counterfactual question answering is lacking. This work aims to bridge this gap. We first derive novel metrics that balance accuracy in factual and counterfactual questions, capturing a more complete view of the reasoning abilities of language models than traditional factual-only based metrics. Second, we propose several fine-tuning approaches that aim to elicit better reasoning mechanisms, in the sense of the proposed metrics. Finally, we evaluate the performance of the fine-tuned language models in a variety of realistic scenarios. In particular, we investigate to what extent our fine-tuning approaches systemically achieve better generalization with respect to the base models in several problems that require, among others, inductive and deductive reasoning capabilities.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.03767>)

[PDF](<https://arxiv.org/pdf/2410.03767.pdf>)
