# TimeSeriesExam: A time series understanding exam

Source: [TimeSeriesExam: A time series understanding exam](<https://www.sota2.com/research/paper/68fa75541e209131cea67378>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:32:00.805Z

Authors: Yifu Cai, Arjun Choudhry, Mononito Goswami, Artur Dubrawski

Published: 2024-10-18T02:37:14.000Z

## Abstract

Large Language Models (LLMs) have recently demonstrated a remarkable ability to model time series data. These capabilities can be partly explained if LLMs understand basic time series concepts. However, our knowledge of what these models understand about time series data remains relatively limited. To address this gap, we introduce TimeSeriesExam, a configurable and scalable multiple-choice question exam designed to assess LLMs across five core time series understanding categories: pattern recognition, noise understanding, similarity analysis, anomaly detection, and causality analysis. TimeSeriesExam comprises of over 700 questions, procedurally generated using 104 carefully curated templates and iteratively refined to balance difficulty and their ability to discriminate good from bad models. We test 7 state-of-the-art LLMs on the TimeSeriesExam and provide the first comprehensive evaluation of their time series understanding abilities. Our results suggest that closed-source models such as GPT-4 and Gemini understand simple time series concepts significantly better than their open-source counterparts, while all models struggle with complex concepts such as causality analysis. We believe that the ability to programatically generate questions is fundamental to assessing and improving LLM's ability to understand and reason about time series data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Time Series Question Answering on TimeSeriesExam synthetic (test)](<https://www.sota2.com/research/sota/time-series-question-answering-on-timeseriesexam-synthetic-test.md>): Time Series Question Answering; TimeSeriesExam synthetic (test); Anomaly Detection: 28.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.14752>)

[PDF](<https://arxiv.org/pdf/2410.14752.pdf>)
