# Differentiable Grammars for Videos

Source: [Differentiable Grammars for Videos](<https://www.sota2.com/research/paper/68fa67041e209131cea13c67>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:04:19.938Z

Authors: AJ Piergiovanni, Anelia Angelova, Michael S. Ryoo

Published: 2019-02-01T18:58:18.000Z

## Abstract

This paper proposes a novel algorithm which learns a formal regular grammar from real-world continuous data, such as videos. Learning latent terminals, non-terminals, and production rules directly from continuous data allows the construction of a generative model capturing sequential structures with multiple possibilities. Our model is fully differentiable, and provides easily interpretable results which are important in order to understand the learned structures. It outperforms the state-of-the-art on several challenging datasets and is more accurate for forecasting future activities in videos. We plan to open-source the code. https://sites.google.com/view/differentiable-grammars

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Temporal Action Localization on MultiTHUMOS](<https://www.sota2.com/research/sota/temporal-action-localization-on-multithumos.md>): Temporal Action Localization; MultiTHUMOS; f-mAP: 48.2.

- [Temporal Action Localization on Charades](<https://www.sota2.com/research/sota/temporal-action-localization-on-charades.md>): Temporal Action Localization; Charades; f-mAP: 22.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1902.00505>)

[PDF](<https://arxiv.org/pdf/1902.00505.pdf>)
