# Learning Lifted Action Models from Unsupervised Visual Traces Source: [Learning Lifted Action Models from Unsupervised Visual Traces]() Collection index: [llms.txt]() Data updated: 2026-05-09T05:00:12.752Z Authors: Kai Xi, Stephen Gould, Sylvie Thi\\'ebaux Published: 2026-04-21T03:49:04.000Z ## Abstract Efficient construction of models capturing the preconditions and effects of actions is essential for applying AI planning in real-world domains. Extensive prior work has explored learning such models from high-level descriptions of state and/or action sequences. In this paper, we tackle a more challenging setting: learning lifted action models from sequences of state images, without action observation. We propose a deep learning framework that jointly learns state prediction, action prediction, and a lifted action model. We also introduce a mixed-integer linear program (MILP) to prevent prediction collapse and self-reinforcing errors among predictions. The MILP takes the predicted states, actions, and action model over a subset of traces and solves for logically consistent states, actions, and action model that are as close as possible to the original predictions. Pseudo-labels extracted from the MILP solution are then used to guide further training. Experiments across multiple domains show that integrating MILP-based correction helps the model escape local optima and converge toward globally consistent solutions. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Action Model Learning on Visual Trace - Blocksworld MNIST grid (test)](): Action Model Learning; Visual Trace - Blocksworld MNIST grid (test); Error: 0. - [Action Model Learning on Visual Trace Gripper (test)](): Action Model Learning; Visual Trace Gripper (test); Error Rate: 0. - [Action Model Learning on Visual Trace Logistics (test)](): Action Model Learning; Visual Trace Logistics (test); Error: 0. - [Action Model Learning on Visual Trace Blocksworld (Synthesized) (test)](): Action Model Learning; Visual Trace Blocksworld (Synthesized) (test); Error: 0. - [Action Model Learning on Visual Trace Hanoi (test)](): Action Model Learning; Visual Trace Hanoi (test); Error: 0. - [Action Model Learning on Visual Trace 8-puzzle (test)](): Action Model Learning; Visual Trace 8-puzzle (test); Error (Err): 0. ## Original sources and implementation [Original paper]() [PDF]()