# Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance Source: [Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance]() Collection index: [llms.txt]() Data updated: 2025-10-26T03:00:03.231Z Authors: Mackenzie J. Meni, Ryan T. White, Michael Mayo, Kevin Pilkiewicz Published: 2023-08-28T23:33:07.000Z ## Abstract Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building and training them are not straightforward processes. To add structure to these efforts, we derive new mathematical results to efficiently measure the changes in entropy as fully-connected and convolutional neural networks process data. By measuring the change in entropy as networks process data effectively, patterns critical to a well-performing network can be visualized and identified. Entropy-based loss terms are developed to improve dense and convolutional model accuracy and efficiency by promoting the ideal entropy patterns. Experiments in image compression, image classification, and image segmentation on benchmark datasets demonstrate these losses guide neural networks to learn rich latent data representations in fewer dimensions, converge in fewer training epochs, and achieve higher accuracy. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()