# Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization

Source: [Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization](<https://www.sota2.com/research/paper/68fa640f1e209131cea026e7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-14T16:52:28.099Z

Authors: Baohan Xu, Yanwei Fu, Yu-Gang Jiang, Boyang Li, Leonid Sigal

Published: 2015-11-16T01:40:15.000Z

## Abstract

Emotion is a key element in user-generated videos. However, it is difficult to understand emotions conveyed in such videos due to the complex and unstructured nature of user-generated content and the sparsity of video frames expressing emotion. In this paper, for the first time, we study the problem of transferring knowledge from heterogeneous external sources, including image and textual data, to facilitate three related tasks in understanding video emotion: emotion recognition, emotion attribution and emotion-oriented summarization. Specifically, our framework (1) learns a video encoding from an auxiliary emotional image dataset in order to improve supervised video emotion recognition, and (2) transfers knowledge from an auxiliary textual corpora for zero-shot recognition of emotion classes unseen during training. The proposed technique for knowledge transfer facilitates novel applications of emotion attribution and emotion-oriented summarization. A comprehensive set of experiments on multiple datasets demonstrate the effectiveness of our framework.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Emotion Recognition on E-6](<https://www.sota2.com/research/sota/video-emotion-recognition-on-e-6.md>): Video Emotion Recognition; E-6; Accuracy: 51.2.

- [Video Emotion Recognition on VE-8 (test)](<https://www.sota2.com/research/sota/video-emotion-recognition-on-ve-8-test.md>): Video Emotion Recognition; VE-8 (test); Accuracy: 44.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1511.04798>)

[PDF](<https://arxiv.org/pdf/1511.04798.pdf>)