# Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Method Source: [Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Method]() Collection index: [llms.txt]() Data updated: 2026-03-28T05:00:04.914Z Authors: Chenxu Yu, Wenqi Fang Published: 2026-03-26T04:42:27.000Z ## Abstract As a representative continuous-depth neural network approach, stochastic differential equation (SDE)-based Bayesian neural networks (BNNs) have attracted considerable attention due to their solid theoretical foundations and strong potential for real-world applications. However, their reliance on numerical SDE solvers inevitably incurs a large number of function evaluations (NFEs), resulting in high computational cost and occasional convergence instability. To address these challenges, we propose a Nesterov-accelerated gradient (NAG) enhanced SDE-BNN model. By integrating NAG into the SDE-BNN framework along with an NFE-dependent residual skip connection, our method accelerates convergence and substantially reduces NFEs during both training and testing. Extensive empirical results show that our model consistently outperforms conventional SDE-BNNs across various tasks, including image classification and sequence modeling, achieving lower NFEs and improved predictive accuracy. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on MNIST (Accuracy, AUC, NLL)](): Image Classification; MNIST; Accuracy: 99.04. - [Image Classification on CIFAR-10 (Accuracy, AUC, NLL)](): Image Classification; CIFAR-10; Accuracy: 88.36. ## Original sources and implementation [Original paper]() [PDF]()