# Entanglement Entropy of Target Functions for Image Classification and Convolutional Neural Network

Source: [Entanglement Entropy of Target Functions for Image Classification and Convolutional Neural Network](<https://www.sota2.com/research/paper/68fa36891e209131ce8aa8f7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:53:58.137Z

Authors: Ya-Hui Zhang

Published: 2017-10-16T05:54:38.000Z

## Abstract

The success of deep convolutional neural network (CNN) in computer vision especially image classification problems requests a new information theory for function of image, instead of image itself. In this article, after establishing a deep mathematical connection between image classification problem and quantum spin model, we propose to use entanglement entropy, a generalization of classical Boltzmann-Shannon entropy, as a powerful tool to characterize the information needed for representation of general function of image. We prove that there is a sub-volume-law bound for entanglement entropy of target functions of reasonable image classification problems. Therefore target functions of image classification only occupy a small subspace of the whole Hilbert space. As a result, a neural network with polynomial number of parameters is efficient for representation of such target functions of image. The concept of entanglement entropy can also be useful to characterize the expressive power of different neural networks. For example, we show that to maintain the same expressive power, number of channels $D$ in a convolutional neural network should scale with the number of convolution layers $n\_c$ as $D\\sim D\_0^{\\frac{1}{n\_c}}$. Therefore, deeper CNN with large $n\_c$ is more efficient than shallow ones.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1710.05520>)

[PDF](<https://arxiv.org/pdf/1710.05520.pdf>)