# DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning

Source: [DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning](<https://www.sota2.com/research/paper/68fa6ba71e209131cea2f998>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:54:05.632Z

Authors: Javier Yu, Joseph A. Vincent, Mac Schwager

Published: 2021-09-17T17:45:36.000Z

## Abstract

We present a distributed algorithm that enables a group of robots to collaboratively optimize the parameters of a deep neural network model while communicating over a mesh network. Each robot only has access to its own data and maintains its own version of the neural network, but eventually learns a model that is as good as if it had been trained on all the data centrally. No robot sends raw data over the wireless network, preserving data privacy and ensuring efficient use of wireless bandwidth. At each iteration, each robot approximately optimizes an augmented Lagrangian function, then communicates the resulting weights to its neighbors, updates dual variables, and repeats. Eventually, all robots' local network weights reach a consensus. For convex objective functions, we prove this consensus is a global optimum. We compare our algorithm to two existing distributed deep neural network training algorithms in (i) an MNIST image classification task, (ii) a multi-robot implicit mapping task, and (iii) a multi-robot reinforcement learning task. In all of our experiments our method out performed baselines, and was able to achieve validation loss equivalent to centrally trained models. See \\href{https://msl.stanford.edu/projects/dist\_nn\_train}{https://msl.stanford.edu/projects/dist\\\_nn\\\_train} for videos and a link to our GitHub repository.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [SST Prediction on OISST Central Atlantic](<https://www.sota2.com/research/sota/sst-prediction-on-oisst-central-atlantic.md>): SST Prediction; OISST Central Atlantic; MSE: 0.175.

- [SST Prediction on OISST North Indian Ocean](<https://www.sota2.com/research/sota/sst-prediction-on-oisst-north-indian-ocean.md>): SST Prediction; OISST North Indian Ocean; MSE: 0.222.

- [SST Prediction on OISST South Indian Ocean](<https://www.sota2.com/research/sota/sst-prediction-on-oisst-south-indian-ocean.md>): SST Prediction; OISST South Indian Ocean; MSE: 0.316.

- [SST Prediction on OISST West Pacific](<https://www.sota2.com/research/sota/sst-prediction-on-oisst-west-pacific.md>): SST Prediction; OISST West Pacific; MSE: 0.166.

- [SST Prediction on OISST East Pacific](<https://www.sota2.com/research/sota/sst-prediction-on-oisst-east-pacific.md>): SST Prediction; OISST East Pacific; MSE: 0.303.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2109.08665>)

[PDF](<https://arxiv.org/pdf/2109.08665.pdf>)
