# MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues Source: [MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues]() Collection index: [llms.txt]() Data updated: 2026-05-23T05:00:04.712Z Authors: Dazhao Du, Liao Duan, Jian Liu, Tao Han, Yujia Zhang, Eric Liu, Xi Chen, Song Guo Published: 2026-05-21T03:40:22.000Z ## Abstract Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed video, is a key test of whether multimodal large language models (MLLMs) understand not only what happens but also when it happens. Although modern MLLMs describe video content fluently, their timestamp predictions remain unreliable, while existing remedies either require costly post-training on temporal annotations or rely on coarse training-free heuristics. In this work, we probe the cross-modal attention of MLLMs and uncover a perception-generation gap. Our key finding is that MLLMs often know the target interval during prefill, but lose this signal when generating the final answer. In the prefill stage, a sparse set of attention heads, which we call \\emph{Temporal Grounding Heads} (TG-Heads), concentrates query-to-video attention on the ground-truth interval. During autoregressive decoding, however, the answer tokens shift attention away from this interval toward visually salient but query-irrelevant segments. This observation motivates an inference-time read-then-regenerate framework. We first convert TG-Head prefill attention into a debiased frame-level relevance signal and extract the high-attention interval it highlights. We then re-invoke the MLLM with visual context restricted to this interval, using video cropping or attention masking to suppress distractors. Without parameter updates and architectural changes, our framework consistently improves MiMo-VL-7B, Qwen3-VL-8B, and TimeLens-8B on three VTG benchmarks, with gains of up to +3.5 mIoU. The project website can be found at https://ddz16.github.io/mllmsknowwhen.github.io/. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Video Temporal Grounding on Charades TimeLens-Bench (test)](): Video Temporal Grounding; Charades TimeLens-Bench (test); R1@0.3: 76.9. - [Video Temporal Grounding on QVHighlights TimeLens-Bench (test)](): Video Temporal Grounding; QVHighlights TimeLens-Bench (test); R1@0.3: 80.6. - [Temporal Video Grounding on ActivityNet TimeLens (test)](): Temporal Video Grounding; ActivityNet TimeLens (test); Recall@0.3: 68.9. ## Original sources and implementation [Original paper]() [PDF]()