# VisionLaw: Inferring Interpretable Intrinsic Dynamics from Visual Observations via Bilevel Optimization

Source: [VisionLaw: Inferring Interpretable Intrinsic Dynamics from Visual Observations via Bilevel Optimization](<https://www.sota2.com/research/paper/68fa7abf1e209131cea84fa6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-14T05:00:11.146Z

Authors: Jiajing Lin, Shu Jiang, Qingyuan Zeng, Zhenzhong Wang, Min Jiang

Published: 2025-08-19T12:52:16.000Z

## Abstract

The intrinsic dynamics of an object governs its physical behavior in the real world, playing a critical role in enabling physically plausible interactive simulation with 3D assets. Existing methods have attempted to infer the intrinsic dynamics of objects from visual observations, but generally face two major challenges: one line of work relies on manually defined constitutive priors, making it difficult to align with actual intrinsic dynamics; the other models intrinsic dynamics using neural networks, resulting in limited interpretability and poor generalization. To address these challenges, we propose VisionLaw, a bilevel optimization framework that infers interpretable expressions of intrinsic dynamics from visual observations. At the upper level, we introduce an LLMs-driven decoupled constitutive evolution strategy, where LLMs are prompted to act as physics experts to generate and revise constitutive laws, with a built-in decoupling mechanism that substantially reduces the search complexity of LLMs. At the lower level, we introduce a vision-guided constitutive evaluation mechanism, which utilizes visual simulation to evaluate the consistency between the generated constitutive law and the underlying intrinsic dynamics, thereby guiding the upper-level evolution. Experiments on both synthetic and real-world datasets demonstrate that VisionLaw can effectively infer interpretable intrinsic dynamics from visual observations. It significantly outperforms existing state-of-the-art methods and exhibits strong generalization for interactive simulation in novel scenarios.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Intrinsic Dynamics Consistency on BouncyBall synthetic (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-bouncyball-synthetic-test.md>): Intrinsic Dynamics Consistency; BouncyBall synthetic (test); Chamfer Distance: 1.08.

- [Intrinsic Dynamics Consistency on ClayCat synthetic (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-claycat-synthetic-test.md>): Intrinsic Dynamics Consistency; ClayCat synthetic (test); Chamfer Distance: 0.77.

- [Intrinsic Dynamics Consistency on HoneyBottle synthetic (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-honeybottle-synthetic-test.md>): Intrinsic Dynamics Consistency; HoneyBottle synthetic (test); Chamfer Distance: 0.79.

- [Intrinsic Dynamics Consistency on JellyDuck synthetic (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-jellyduck-synthetic-test.md>): Intrinsic Dynamics Consistency; JellyDuck synthetic (test); Chamfer Distance: 5.19.

- [Intrinsic Dynamics Consistency on RubberPawn synthetic (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-rubberpawn-synthetic-test.md>): Intrinsic Dynamics Consistency; RubberPawn synthetic (test); Chamfer Distance: 0.94.

- [Intrinsic Dynamics Consistency on Synthetic Datasets Average (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-synthetic-datasets-average-test.md>): Intrinsic Dynamics Consistency; Synthetic Datasets Average (test); Chamfer Distance: 1.65.

- [Intrinsic Dynamics Consistency on SandFish synthetic (test)](<https://www.sota2.com/research/sota/intrinsic-dynamics-consistency-on-sandfish-synthetic-test.md>): Intrinsic Dynamics Consistency; SandFish synthetic (test); Chamfer Distance: 1.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2508.13792>)

[PDF](<https://arxiv.org/pdf/2508.13792.pdf>)
