# RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression Source: [RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression]() Collection index: [llms.txt]() Data updated: 2026-01-04T04:19:55.525Z Authors: Uri Gadot, Assaf Shocher, Shie Mannor, Gal Chechik, Assaf Hallak Published: 2025-01-21T15:36:08.000Z ## Abstract Video encoders optimize compression for human perception by minimizing reconstruction error under bit-rate constraints. In many modern applications such as autonomous driving, an overwhelming majority of videos serve as input for AI systems performing tasks like object recognition or segmentation, rather than being watched by humans. It is therefore useful to optimize the encoder for a downstream task instead of for perceptual image quality. However, a major challenge is how to combine such downstream optimization with existing standard video encoders, which are highly efficient and popular. Here, we address this challenge by controlling the Quantization Parameters (QPs) at the macro-block level to optimize the downstream task. This granular control allows us to prioritize encoding for task-relevant regions within each frame. We formulate this optimization problem as a Reinforcement Learning (RL) task, where the agent learns to balance long-term implications of choosing QPs on both task performance and bit-rate constraints. Notably, our policy does not require the downstream task as an input during inference, making it suitable for streaming applications and edge devices such as vehicles. We demonstrate significant improvements in two tasks, car detection, and ROI (saliency) encoding. Our approach improves task performance for a given bit rate compared to traditional task agnostic encoding methods, paving the way for more efficient task-aware video compression. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Object Detection on BDD100K (test) using Precision/Recall Rate Metrics](): Object Detection; BDD100K (test); Precision (low-rate): 0.36. - [Video Compression on BDD100K (test)](): Video Compression; BDD100K (test); PSNR (low-rate): 29.03. - [Semantic Segmentation on BDD100K (test)](): Semantic segmentation; BDD100K (test); mIoU: not recorded. ## Original sources and implementation [Original paper]() [PDF]()