# Sparse Flows: Pruning Continuous-depth Models

Source: [Sparse Flows: Pruning Continuous-depth Models](<https://www.sota2.com/research/paper/68fa6b421e209131cea2d093>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:44:09.187Z

Authors: Lucas Liebenwein, Ramin Hasani, Alexander Amini, Daniela Rus

Published: 2021-06-24T01:40:17.000Z

## Abstract

Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative modeling as continuous normalizing flows. In this work, we design a framework to decipher the internal dynamics of these continuous depth models by pruning their network architectures. Our empirical results suggest that pruning improves generalization for neural ODEs in generative modeling. We empirically show that the improvement is because pruning helps avoid mode-collapse and flatten the loss surface. Moreover, pruning finds efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy. We hope our results will invigorate further research into the performance-size trade-offs of modern continuous-depth models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Gene Regulatory Network Inference on SIM350 5% noise (test)](<https://www.sota2.com/research/sota/gene-regulatory-network-inference-on-sim350-5-percent-noise-test.md>): Gene regulatory network inference; SIM350 5% noise (test); Sparsity: 96.

- [Gene Regulatory Network Inference on Breast cancer in pseudotime](<https://www.sota2.com/research/sota/gene-regulatory-network-inference-on-breast-cancer-in-pseudotime.md>): Gene regulatory network inference; Breast cancer in pseudotime; Sparsity: 95.7.

- [Gene Regulatory Network Inference on Yeast cell cycle](<https://www.sota2.com/research/sota/gene-regulatory-network-inference-on-yeast-cell-cycle.md>): Gene regulatory network inference; Yeast cell cycle; Sparsity: 95.22.

- [Gene regulatory dynamics prediction on SIM350 5% noise (test)](<https://www.sota2.com/research/sota/gene-regulatory-dynamics-prediction-on-sim350-5-percent-noise-test.md>): Gene regulatory dynamics prediction; SIM350 5% noise (test); MSE: 3.6.

- [Gene expression dynamics prediction on Hematopoiesis Erythroid lineage (test)](<https://www.sota2.com/research/sota/gene-expression-dynamics-prediction-on-hematopoiesis-erythroid-lineage-test.md>): Gene expression dynamics prediction; Hematopoesis Erythroid lineage (test); Sparsity: 0.9444.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2106.12718>)

[PDF](<https://arxiv.org/pdf/2106.12718.pdf>)