# SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting Source: [SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting]() Collection index: [llms.txt]() Data updated: 2026-04-15T05:00:04.591Z Authors: Yicheng Deng, Hideaki Hayashi, Hajime Nagahara Published: 2024-07-30T13:02:08.000Z ## Abstract Facial expression spotting, identifying periods where facial expressions occur in a video, is a significant yet challenging task in facial expression analysis. The issues of irrelevant facial movements and the challenge of detecting subtle motions in micro-expressions remain unresolved, hindering accurate expression spotting. In this paper, we propose an efficient framework for facial expression spotting. First, we propose a Compact Sliding-Window-based Multi-temporal-Resolution Optical flow (CSW-MRO) feature, which calculates multi-temporal-resolution optical flow of the input image sequence within compact sliding windows. The window length is tailored to perceive complete micro-expressions and distinguish between general macro- and micro-expressions. CSW-MRO can effectively reveal subtle motions while avoiding the optical flow being dominated by head movements. Second, we propose SpotFormer, a multi-scale spatio-temporal Transformer that simultaneously encodes spatio-temporal relationships of the CSW-MRO features for accurate frame-level probability estimation. In SpotFormer, we use the proposed Facial Local Graph Pooling (FLGP) operation and convolutional layers to extract multi-scale spatio-temporal features. We show the validity of the architecture of SpotFormer by comparing it with several model variants. Third, we introduce supervised contrastive learning into SpotFormer to enhance the discriminability between different types of expressions. Extensive experiments on SAMM-LV, CAS(ME)^2, and CAS(ME)^3 show that our method outperforms state-of-the-art models, particularly in micro-expression spotting. Code is available at https://github.com/KinopioIsAllIn/SpotFormer. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Facial Expression Spotting on CAS(ME)2](): Facial Expression Spotting; CAS(ME)2; Overall F1-score: 48.41. - [Facial expression spotting on SAMM-LV](): Facial Expression Spotting; SAMM-LV; ME F1-score: 42.81. - [Facial Expression Spotting on CAS(ME)3 82](): Facial Expression Spotting; CAS(ME)3 82; ME F1-score: 20.37. - [Spotting on SAMMLV](): Spotting; SAMMLV; F1 Score: 44.5. ## Original sources and implementation [Original paper]() [PDF]()