# Know Your Attention Maps: Class-specific Token Masking for Weakly Supervised Semantic Segmentation Source: [Know Your Attention Maps: Class-specific Token Masking for Weakly Supervised Semantic Segmentation]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:56:50.936Z Authors: Joelle Hanna, Damian Borth Published: 2025-07-09T13:53:34.000Z ## Abstract Weakly Supervised Semantic Segmentation (WSSS) is a challenging problem that has been extensively studied in recent years. Traditional approaches often rely on external modules like Class Activation Maps to highlight regions of interest and generate pseudo segmentation masks. In this work, we propose an end-to-end method that directly utilizes the attention maps learned by a Vision Transformer (ViT) for WSSS. We propose training a sparse ViT with multiple \[CLS\] tokens (one for each class), using a random masking strategy to promote \[CLS\] token - class assignment. At inference time, we aggregate the different self-attention maps of each \[CLS\] token corresponding to the predicted labels to generate pseudo segmentation masks. Our proposed approach enhances the interpretability of self-attention maps and ensures accurate class assignments. Extensive experiments on two standard benchmarks and three specialized datasets demonstrate that our method generates accurate pseudo-masks, outperforming related works. Those pseudo-masks can be used to train a segmentation model which achieves results comparable to fully-supervised models, significantly reducing the need for fine-grained labeled data. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Semantic Segmentation on COCO (val)](): Semantic segmentation; COCO (val); mIoU: 43.2. - [Semantic Segmentation on COCO 2014 (val)](): Semantic segmentation; COCO 2014 (val); mIoU: 43.2. - [Semantic Segmentation on PASCAL VOC 2012 (test)](): Semantic segmentation; PASCAL VOC 2012 (test); mIoU: 73.5. - [Semantic Segmentation on PASCAL VOC 2012 (val)](): Semantic segmentation; PASCAL VOC 2012 (val); Mean IoU: 72.7. ## Original sources and implementation [Original paper]() [PDF]()