distributed-deep-learning

Building Large-Scale AI Applications for Distributed Big Data

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

Distributed Keras Engine, Make Keras faster with only one line of code.

Learn applied deep learning from zero to deployment using TensorFlow 1.8+

sensAI: ConvNets Decomposition via Class Parallelism for Fast Inference on Live Data

A Portable C Library for Distributed CNN Inference on IoT Edge Clusters

TensorFlow (1.8+) Datasets, Feature Columns, Estimators and Distributed Training using Google Cloud Machine Learning Engine

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.

WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.

Eager-SGD is a decentralized asynchronous SGD. It utilizes novel partial collectives operations to accumulate the gradients across all the processes.

Java based Convolutional Neural Network package running on Apache Spark framework

Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.

Distributed deep learning framework based on pytorch/mxnet/numba and nccl.

SHUKUN Technology Co.,Ltd Algorithm intern (2020/12-2021/5). Multi-GPU, Multi-node training for deep learning models. Horovod, NVIDIA clara train sdk, configuration tutorial,performance testing.

Collection of resources for automatic deployment of distributed deep learning jobs on a Kubernetes cluster

Distributed Deep Reinforcement Learning for Large Scale Robotic Simulations 👨‍💻🤖🕸🕹🕷❤️👨‍🔬

PyTorch Examples for Beginners

Horovod Tutorial for Pytorch using NVIDIA-Docker.

An implementation of a distributed ResNet model for classifying CIFAR-10 and MNIST datasets.

Yelp review classification using CNN model with horovod on HPC cluster

mnist, using caffe and openmpi

Distributed Tensorflow, Keras and BigDL on Apache Spark

Java based Convolutional Neural Network package running on Apache Spark framework

Simultaneous Multi-Party Learning Framework