Automated Gaze-based Behavioral Segmentation and Temporal Representation for Bridge Inspection in Unconstrained 3D Environments
About
Visual bridge inspection is a knowledge-intensive task in which inspectors coordinate visual search, spatial navigation, structural reasoning, and defect identification and documentation. It is a central maintenance task for bridges and a key basis for safety assessments, yet its results are susceptible to individual subjectivity. While eye-tracking-based behavioral studies quantify underlying processes, existing research often imposes restrictive simplifications to reduce environmental complexity, thereby compromising ecological validity. This study proposes an automated data analytics framework for converting multimodal inspection data into an inspection mode time series. Unconstrained 3D gaze, head-movement, drone navigation, and scene geometry data are segmented into temporal windows and classified into three functional modes: global scanning, local inspection, and navigation. The resulting temporal representation enables the extraction of interpretable behavioral descriptors, including transition probabilities, dwell times, transition entropy, fixation measures, and spatial revisit metrics. A feasibility study using a virtual bridge inspection platform demonstrates that the proposed representation captures meaningful differences in inspection strategy and reveals exploratory relationships with inspection performance. This study contributes a framework for human-informed computer-aided infrastructure inspection systems, inspector training, and data-driven assessment of constructed facilities.