Adaptive Consensus ADMM for Distributed Optimization

About

The alternating direction method of multipliers (ADMM) is commonly used for distributed model fitting problems, but its performance and reliability depend strongly on user-defined penalty parameters. We study distributed ADMM methods that boost performance by using different fine-tuned algorithm parameters on each worker node. We present a O(1/k) convergence rate for adaptive ADMM methods with node-specific parameters, and propose adaptive consensus ADMM (ACADMM), which automatically tunes parameters without user oversight.

Zheng Xu, Gavin Taylor, Hao Li, Mario Figueiredo, Xiaoming Yuan, Tom Goldstein · 2017

Related benchmarks

Task	Dataset	Result
Elastic net regression	Synthetic Elastic net regression	Runtime (s)623	69
Elastic net regression	synthetic 2	Iterations57	5
Elastic net regression	MNIST	Iterations14	5
Elastic net regression	news20	Iterations78	5
Elastic net regression	RCV1	Iterations8	5
Elastic net regression	Realsim	Iterations9	5
Semidefinite programming	Ham-9-5-6	Iterations30	5
Sparse Logistic Regression	synthetic 1	Iterations24	5
Sparse Logistic Regression	synthetic 2	Iterations114	5
Sparse Logistic Regression	MNIST	Iterations149	5