DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Python43,191 stars5,025 forksApache-2.0 billion-parameterscompressiondata-parallelismdeep-learninggpuinferencemachine-learningmixture-of-expertsmodel-parallelismpipeline-parallelismpytorchtrillion-parameterszero