# SegCodeNet: Color-Coded Segmentation Masks for Activity Detection from Wearable Cameras

Source: [SegCodeNet: Color-Coded Segmentation Masks for Activity Detection from Wearable Cameras](<https://www.sota2.com/research/paper/68fa682a1e209131cea1a733>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:30:00.158Z

Authors: Asif Shahriyar Sushmit, Partho Ghosh, Md.Abrar Istiak, Nayeeb Rashid, Ahsan Habib Akash, Taufiq Hasan

Published: 2020-08-19T14:01:53.000Z

## Abstract

Activity detection from first-person videos (FPV) captured using a wearable camera is an active research field with potential applications in many sectors, including healthcare, law enforcement, and rehabilitation. State-of-the-art methods use optical flow-based hybrid techniques that rely on features derived from the motion of objects from consecutive frames. In this work, we developed a two-stream network, the \\emph{SegCodeNet}, that uses a network branch containing video-streams with color-coded semantic segmentation masks of relevant objects in addition to the original RGB video-stream. We also include a stream-wise attention gating that prioritizes between the two streams and a frame-wise attention module that prioritizes the video frames that contain relevant features. Experiments are conducted on an FPV dataset containing $18$ activity classes in office environments. In comparison to a single-stream network, the proposed two-stream method achieves an absolute improvement of $14.366\\%$ and $10.324\\%$ for averaged F1 score and accuracy, respectively, when average results are compared for three different frame sizes $224\\times224$, $112\\times112$, and $64\\times64$. The proposed method provides significant performance gains for lower-resolution images with absolute improvements of $17\\%$ and $26\\%$ in F1 score for input dimensions of $112\\times112$ and $64\\times64$, respectively. The best performance is achieved for a frame size of $224\\times224$ yielding an F1 score and accuracy of $90.176\\%$ and $90.799\\%$ which outperforms the state-of-the-art Inflated 3D ConvNet (I3D) \\cite{carreira2017quo} method by an absolute margin of $4.529\\%$ and $2.419\\%$, respectively.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2008.08452>)

[PDF](<https://arxiv.org/pdf/2008.08452.pdf>)
