distributed-optimization

Federated Learning Library: https://fedml.ai

Federated Optimization in Heterogeneous Networks (MLSys '20)

All materials you need for Federated Learning: blogs, videos, papers, and softwares, etc.

Fair Resource Allocation in Federated Learning (ICLR '20)

FedTorch is a generic repository for benchmarking different federated and distributed learning algorithms using PyTorch Distributed API.

Distributed Linear Programming Solver on top of Apache Spark

DISROPT: A Python framework for distributed optimization

Implementation of (overlap) local SGD in Pytorch

FedDANE: A Federated Newton-Type Method (Asilomar Conference on Signals, Systems, and Computers ‘19)

This library is an implementation of the algorithm described in Distributed Trajectory Estimation with Privacy and Communication Constraints: a Two-Stage Distributed Gauss-Seidel Approach.

Scalable, structured, dynamically-scheduled hyperparameter optimization.

Communication-efficient decentralized SGD (Pytorch)

tvopt is a prototyping and benchmarking Python framework for time-varying (or online) optimization.

Implementation of Redundancy Infused SGD for faster distributed SGD.

Implementation of Local Updates Periodic Averaging (LUPA) SGD

We present an algorithm to dynamically adjust the data assigned for each worker at every epoch during the training in a heterogeneous cluster. We empirically evaluate the performance of the dynamic partitioning by training deep neural networks on the CIFAR10 dataset.

Implementation of consensus algorithms using row-stochastic weights over directed graphs

We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while maintaining the performance of vanilla SGD. We empirically evaluate the performance of the compression methods by training deep neural networks on the CIFAR10 dataset.

optopy is a prototyping and benchmarking Python framework for optimization, both static and dynamic, centralized and distributed

MATLAB implementation of the paper "Distributed Optimization of Average Consensus Containment with Multiple Stationary Leaders" [arXiv 2022].

This repository contains the code that produces the numeric section in On the Use of TensorFlow Computation Graphs in combination with Distributed Optimization to Solve Large-Scale Convex Problems

Error feedback based quantization aided and convergence guaranteed Communication Efficient Federated Linear and Deep GCCA

Project work of Distributed Control System

Error feedback based quantization aided and convergence guaranteed Communication Efficient Federated Linear and Deep GCCA

Pomodoro: Progressive Decomposition Methods with Acceleration

Parallel optimizer based on the Global Function Search

We present UDP-based aggregation algorithms for federated learning. We also present a scalable framework for practical federated learning. We empirically evaluate the performance by training deep convolutional neural networks on the MNIST dataset and the CIFAR10 dataset.