# Mind the Gap: Removing the Discretization Gap in Differentiable Logic Gate Networks

Source: [Mind the Gap: Removing the Discretization Gap in Differentiable Logic Gate Networks](<https://www.sota2.com/research/paper/68fa79451e209131cea7cfb0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-11-01T05:00:42.467Z

Authors: Shakir Yousefi, Andreas Plesner, Till Aczel, Roger Wattenhofer

Published: 2025-06-09T07:25:51.000Z

## Abstract

Modern neural networks demonstrate state-of-the-art performance on numerous existing benchmarks; however, their high computational requirements and energy consumption prompt researchers to seek more efficient solutions for real-world deployment. Logic gate networks (LGNs) learns a large network of logic gates for efficient image classification. However, learning a network that can solve a simple problem like CIFAR-10 can take days to weeks to train. Even then, almost half of the network remains unused, causing a discretization gap. This discretization gap hinders real-world deployment of LGNs, as the performance drop between training and inference negatively impacts accuracy. We inject Gumbel noise with a straight-through estimator during training to significantly speed up training, improve neuron utilization, and decrease the discretization gap. We theoretically show that this results from implicit Hessian regularization, which improves the convergence properties of LGNs. We train networks $4.5 \\times$ faster in wall-clock time, reduce the discretization gap by $98\\%$, and reduce the number of unused gates by $100\\%$.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.07500>)

[PDF](<https://arxiv.org/pdf/2506.07500.pdf>)
