# 3DSPA: A 3D Semantic Point Autoencoder for Evaluating Video Realism

Source: [3DSPA: A 3D Semantic Point Autoencoder for Evaluating Video Realism](<https://www.sota2.com/research/paper/699e820c6017c4460ffacf16>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:00:04.574Z

Authors: Bhavik Chandna, Kelsey R. Allen

Published: 2026-02-23T21:00:48.000Z

## Abstract

AI video generation is evolving rapidly. For video generators to be useful for applications ranging from robotics to film-making, they must consistently produce realistic videos. However, evaluating the realism of generated videos remains a largely manual process -- requiring human annotation or bespoke evaluation datasets which have restricted scope. Here we develop an automated evaluation framework for video realism which captures both semantics and coherent 3D structure and which does not require access to a reference video. Our method, 3DSPA, is a 3D spatiotemporal point autoencoder which integrates 3D point trajectories, depth cues, and DINO semantic features into a unified representation for video evaluation. 3DSPA models how objects move and what is happening in the scene, enabling robust assessments of realism, temporal consistency, and physical plausibility. Experiments show that 3DSPA reliably identifies videos which violate physical laws, is more sensitive to motion artifacts, and aligns more closely with human judgments of video quality and realism across multiple datasets. Our results demonstrate that enriching trajectory-based representations with 3D semantics offers a stronger foundation for benchmarking generative video models, and implicitly captures physical rule violations. The code and pretrained model weights will be available at https://github.com/TheProParadox/3dspa\_code.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Detecting physically implausible events on IntPhys2](<https://www.sota2.com/research/sota/detecting-physically-implausible-events-on-intphys2.md>): Detecting physically implausible events; IntPhys2; Permanence (Fixed) Win Rate: 0.7692.

- [Video Physical Commonsense Evaluation on VideoPhy-2](<https://www.sota2.com/research/sota/video-physical-commonsense-evaluation-on-videophy-2.md>): Video Physical Commonsense Evaluation; VideoPhy 2; Spearman PC: 0.74.

- [3D Point Tracking on TAPVid-3D Aria (minival)](<https://www.sota2.com/research/sota/3d-point-tracking-on-tapvid-3d-aria-minival.md>): 3D Point Tracking; TAPVid-3D Aria (minival); 3D-AJ: 17.7.

- [3D Point Tracking on TAPVid-3D Average (minival)](<https://www.sota2.com/research/sota/3d-point-tracking-on-tapvid-3d-average-minival.md>): 3D Point Tracking; TAPVid-3D Average (minival); 3D AJ: 0.14.

- [3D Point Tracking on TAPVid-3D DriveTrack (minival)](<https://www.sota2.com/research/sota/3d-point-tracking-on-tapvid-3d-drivetrack-minival.md>): 3D Point Tracking; TAPVid-3D DriveTrack (minival); 3D AJ Score: 11.9.

- [3D Point Tracking on TAPVid-3D PStudio (minival)](<https://www.sota2.com/research/sota/3d-point-tracking-on-tapvid-3d-pstudio-minival.md>): 3D Point Tracking; TAPVid-3D PStudio (minival); 3D-AJ: 12.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2602.20354>)

[PDF](<https://arxiv.org/pdf/2602.20354.pdf>)