# SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics

Source: [SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics](<https://www.sota2.com/research/paper/69730077b2be0ed85a78c981>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-24T05:00:05.231Z

Authors: Silvia Casola, Ryan Soh-Eun Shim, Felicia K\\"orner, Yuchen Mao, Barbara Plank

Published: 2026-01-22T09:49:29.000Z

## Abstract

An increasing body of work has leveraged multilingual language models for Natural Language Generation tasks such as summarization. A major empirical bottleneck in this area is the shortage of accurate and robust evaluation metrics for many languages, which hinders progress. Recent studies suggest that multilingual language models often use English as an internal pivot language, and that misalignment with this pivot can lead to degraded downstream performance. Motivated by the hypothesis that this mismatch could also apply to multilingual neural metrics, we ask whether steering their activations toward an English pivot can improve correlation with human judgments. We experiment with encoder- and decoder-based metrics and find that test-time intervention methods are effective across the board, increasing metric effectiveness for diverse languages.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Translation Quality Estimation (Coherence) on WMT DA 2022 (test)](<https://www.sota2.com/research/sota/machine-translation-quality-estimation-coherence-on-wmt-da-2022-test.md>): Machine Translation Quality Estimation (Coherence); WMT DA 2022 (test); QE Score (ar): 0.16.

- [Machine Translation Quality Estimation (Completeness) on WMT DA 2022 (test)](<https://www.sota2.com/research/sota/machine-translation-quality-estimation-completeness-on-wmt-da-2022-test.md>): Machine Translation Quality Estimation (Completeness); WMT DA 2022 (test); Completeness Score (AR): 33.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.15809>)

[PDF](<https://arxiv.org/pdf/2601.15809.pdf>)