# Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting

Source: [Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting](<https://www.sota2.com/research/paper/68fa6be01e209131cea30cb0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:59:05.045Z

Authors: Dazhao Du, Bing Su, Zhewei Wei

Published: 2022-02-23T08:49:35.000Z

## Abstract

Transformer-based methods have shown great potential in long-term time series forecasting. However, most of these methods adopt the standard point-wise self-attention mechanism, which not only becomes intractable for long-term forecasting since its complexity increases quadratically with the length of time series, but also cannot explicitly capture the predictive dependencies from contexts since the corresponding key and value are transformed from the same point. This paper proposes a predictive Transformer-based model called {\\em Preformer}. Preformer introduces a novel efficient {\\em Multi-Scale Segment-Correlation} mechanism that divides time series into segments and utilizes segment-wise correlation-based attention for encoding time series. A multi-scale structure is developed to aggregate dependencies at different temporal scales and facilitate the selection of segment length. Preformer further designs a predictive paradigm for decoding, where the key and value come from two successive segments rather than the same segment. In this way, if a key segment has a high correlation score with the query segment, its successive segment contributes more to the prediction of the query segment. Extensive experiments demonstrate that our Preformer outperforms other Transformer-based methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hourly Heat Demand Forecasting on DMA A](<https://www.sota2.com/research/sota/hourly-heat-demand-forecasting-on-dma-a.md>): Hourly heat demand forecasting; DMA A; MAE: 320.68.

- [Hourly Heat Demand Forecasting on DMA B](<https://www.sota2.com/research/sota/hourly-heat-demand-forecasting-on-dma-b.md>): Hourly heat demand forecasting; DMA B; MAE: 627.2.

- [Hourly heat demand forecasting on DMA C](<https://www.sota2.com/research/sota/hourly-heat-demand-forecasting-on-dma-c.md>): Hourly heat demand forecasting; DMA C; MAE: 552.04.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2202.11356>)

[PDF](<https://arxiv.org/pdf/2202.11356.pdf>)
