# Test-Time Alignment for Large Language Models via Textual Model Predictive Control

Source: [Test-Time Alignment for Large Language Models via Textual Model Predictive Control](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-24T05:00:03.763Z

Authors: Kuang-Da Wang, Teng-Ruei Chen, Yu Heng Hung, Guo-Xun Ko, Shuoyang Ding, Yueh-Hua Wu, Yu-Chiang Frank Wang, Chao-Han Huck Yang, Wen-Chih Peng, Ping-Chun Hsieh

Published: 2025-02-28T07:24:33.000Z

## Abstract

Aligning Large Language Models (LLMs) with human preferences through finetuning is resource-intensive, motivating lightweight alternatives at test time. We address test-time alignment through the lens of sequential decision making, a perspective that reveals two fundamental challenges. When actions are defined at the token level, as in guided decoding, alignment suffers from the curse of horizon. Conversely, when actions are at the response level, as in traditional iterative refinement, the curse of dimensionality emerges. To resolve this trade-off, we draw inspiration from Model Predictive Control (MPC) in control theory to propose Textual Model Predictive Control (TMPC), a novel predictive planning framework adapted for aligning LLMs at inference time. A key limitation of standard MPC is its reliance on predefined, hard segment boundaries, which are often absent in text generation. TMPC overcomes this by introducing two principles inspired by hierarchical reinforcement learning: (1) Hindsight Subgoal Identification, where TMPC analyzes generation subgoals to retrospectively identify high-reward intermediate outputs as subgoals. This allows the framework to discover meaningful, task-specific planning steps (e.g., a sentence in machine translation or a bug fix in code generation.). (2) Subgoal-Conditioned Re-Generation, where these identified subgoals are used to guide subsequent planning iterations. By conditioning on these proven, high-quality subgoals, TMPC ensures stable improvement by building upon previously validated successes. TMPC is evaluated on three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis. The results demonstrate that TMPC consistently improves performance, highlighting the generality.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Translation on WMT literary translation (zh→en) 24](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-en-24.md>): Machine Translation; WMT literary translation (zh→en) 24; SEGALE Comet Score: 94.62.

- [Machine Translation on WMT literary translation (zh→ru) 24](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-ru-24.md>): Machine Translation; WMT literary translation (zh→ru) 24; SEGALE\_comet: 91.53.

- [Machine Translation on WMT literary translation (zh→de) 24](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-de-24.md>): Machine Translation; WMT literary translation (zh→de) 24; SEGALE-COMET Score: 91.73.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2502.20795>)

[PDF](<https://arxiv.org/pdf/2502.20795.pdf>)
