# Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmark

Source: [Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmark](<https://www.sota2.com/research/paper/68fa76011e209131cea6adec>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:45:34.560Z

Authors: Joseph Heyward, Jo\\~ao Carreira, Dima Damen, Andrew Zisserman, Viorica P\\u{a}tr\\u{a}ucean

Published: 2024-11-29T18:57:25.000Z

## Abstract

Following the successful 2023 edition, we organised the Second Perception Test challenge as a half-day workshop alongside the IEEE/CVF European Conference on Computer Vision (ECCV) 2024, with the goal of benchmarking state-of-the-art video models and measuring the progress since last year using the Perception Test benchmark. This year, the challenge had seven tracks (up from six last year) and covered low-level and high-level tasks, with language and non-language interfaces, across video, audio, and text modalities; the additional track covered hour-long video understanding and introduced a novel video QA benchmark 1h-walk VQA. Overall, the tasks in the different tracks were: object tracking, point tracking, temporal action localisation, temporal sound localisation, multiple-choice video question-answering, grounded video question-answering, and hour-long video question-answering. We summarise in this report the challenge tasks and results, and introduce in detail the novel hour-long video QA benchmark 1h-walk VQA.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2411.19941>)

[PDF](<https://arxiv.org/pdf/2411.19941.pdf>)