# High-Capacity Expert Binary Networks

Source: [High-Capacity Expert Binary Networks](<https://www.sota2.com/research/paper/68fa699a1e209131cea2302d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:03:46.301Z

Authors: Adrian Bulat, Brais Martinez, Georgios Tzimiropoulos

Published: 2020-10-07T17:58:10.000Z

## Abstract

Network binarization is a promising hardware-aware direction for creating efficient deep models. Despite its memory and computational advantages, reducing the accuracy gap between binary models and their real-valued counterparts remains an unsolved challenging research problem. To this end, we make the following 3 contributions: (a) To increase model capacity, we propose Expert Binary Convolution, which, for the first time, tailors conditional computing to binary networks by learning to select one data-specific expert binary filter at a time conditioned on input features. (b) To increase representation capacity, we propose to address the inherent information bottleneck in binary networks by introducing an efficient width expansion mechanism which keeps the binary operations within the same budget. (c) To improve network design, we propose a principled binary network growth mechanism that unveils a set of network topologies of favorable properties. Overall, our method improves upon prior work, with no increase in computational cost, by $\\sim6 \\%$, reaching a groundbreaking $\\sim 71\\%$ on ImageNet classification. Code will be made available $\\href{https://www.adrianbulat.com/binary-networks}{here}$.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.03558>)

[PDF](<https://arxiv.org/pdf/2010.03558.pdf>)
