# PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation

Source: [PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation](<https://www.sota2.com/research/paper/68fa6c0e1e209131cea31c91>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:10:08.074Z

Authors: Zhe Hu, Hou Pong Chan, Jiachen Liu, Xinyan Xiao, Hua Wu, Lifu Huang

Published: 2022-03-17T05:52:35.000Z

## Abstract

Despite recent progress of pre-trained language models on generating fluent text, existing methods still suffer from incoherence problems in long-form text generation tasks that require proper content control and planning to form a coherent high-level logical flow. In this work, we propose PLANET, a novel generation framework leveraging autoregressive self-attention mechanism to conduct content planning and surface realization dynamically. To guide the generation of output sentences, our framework enriches the Transformer decoder with latent representations to maintain sentence-level semantic plans grounded by bag-of-words. Moreover, we introduce a new coherence-based contrastive learning objective to further improve the coherence of output. Extensive experiments are conducted on two challenging long-form text generation tasks including counterargument generation and opinion article generation. Both automatic and human evaluations show that our method significantly outperforms strong baselines and generates more coherent texts with richer contents.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Argument Generation on ArgGen](<https://www.sota2.com/research/sota/argument-generation-on-arggen.md>): Argument Generation; ArgGen; BLEU-2: 38.55.

- [Opinion Article Generation on OpinionGen](<https://www.sota2.com/research/sota/opinion-article-generation-on-opiniongen.md>): Opinion Article Generation; OpinionGen; BLEU-2: 36.79.

- [Opinion Generation on OpinionGen](<https://www.sota2.com/research/sota/opinion-generation-on-opiniongen.md>): Opinion Generation; OpinionGen; Relevance: 3.89.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2203.09100>)

[PDF](<https://arxiv.org/pdf/2203.09100.pdf>)