# Generalized BackPropagation, \\'{E}tude De Cas: Orthogonality

Source: [Generalized BackPropagation, \\'{E}tude De Cas: Orthogonality](<https://www.sota2.com/research/paper/68fa62ed1e209131ce9fbec4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:29:18.716Z

Authors: Mehrtash Harandi, Basura Fernando

Published: 2016-11-17T22:55:09.000Z

## Abstract

This paper introduces an extension of the backpropagation algorithm that enables us to have layers with constrained weights in a deep network. In particular, we make use of the Riemannian geometry and optimization techniques on matrix manifolds to step outside of normal practice in training deep networks, equipping the network with structures such as orthogonality or positive definiteness. Based on our development, we make another contribution by introducing the Stiefel layer, a layer with orthogonal weights. Among various applications, Stiefel layers can be used to design orthogonal filter banks, perform dimensionality reduction and feature extraction. We demonstrate the benefits of having orthogonality in deep networks through a broad set of experiments, ranging from unsupervised feature learning to fine-grained image classification.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1611.05927>)

[PDF](<https://arxiv.org/pdf/1611.05927.pdf>)