# Patchwork: A Patch-wise Attention Network for Efficient Object Detection and Segmentation in Video Streams Source: [Patchwork: A Patch-wise Attention Network for Efficient Object Detection and Segmentation in Video Streams]() Collection index: [llms.txt]() Data updated: 2025-10-25T21:44:30.600Z Authors: Yuning Chai Published: 2019-04-03T05:58:42.000Z ## Abstract Recent advances in single-frame object detection and segmentation techniques have motivated a wide range of works to extend these methods to process video streams. In this paper, we explore the idea of hard attention aimed for latency-sensitive applications. Instead of reasoning about every frame separately, our method selects and only processes a small sub-window of the frame. Our technique then makes predictions for the full frame based on the sub-windows from previous frames and the update from the current sub-window. The latency reduction by this hard attention mechanism comes at the cost of degraded accuracy. We made two contributions to address this. First, we propose a specialized memory cell that recovers lost context when processing sub-windows. Secondly, we adopt a Q-learning-based policy training strategy that enables our approach to intelligently select the sub-windows such that the staleness in the memory hurts the performance the least. Our experiments suggest that our approach reduces the latency by approximately four times without significantly sacrificing the accuracy on the ImageNet VID video object detection dataset and the DAVIS video object segmentation dataset. We further demonstrate that we can reinvest the saved computation into other parts of the network, and thus resulting in an accuracy increase at a comparable computational cost as the original system and beating other recently proposed state-of-the-art methods in the low latency range. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Video Object Detection on ImageNet VID (val)](): Video Object Detection; ImageNet VID (val); mAP (%): 57.4. ## Original sources and implementation [Original paper]() [PDF]()