Local stochastic ADMM for communication-efficient distributed learning
Ben Issaid, Chaouki; Elgabli, Anis; Bennis, Mehdi (2022-05-16)
C. ben Issaid, A. Elgabli and M. Bennis, "Local Stochastic ADMM for Communication-Efficient Distributed Learning," 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA, 2022, pp. 1880-1885, doi: 10.1109/WCNC51071.2022.9771559
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
https://urn.fi/URN:NBN:fi-fe2023020926609
Tiivistelmä
Abstract
In this paper, we propose a communication-efficient alternating direction method of multipliers (ADMM)-based algorithm for solving a distributed learning problem in the stochastic non-convex setting. Our approach runs a few stochastic gradient descent (SGD) steps to solve the local problem at each worker instead of finding the exact/approximate solution as proposed by existing ADMM-based works. By doing so, the proposed framework strikes a good balance between the computation and communication costs. Extensive simulation results show that our algorithm significantly outperforms existing stochastic ADMM in terms of communication-efficiency, notably in the presence of non-independent and identically distributed (non-IID) data.
Kokoelmat
- Avoin saatavuus [32026]