research
∙
04/11/2023
TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Training
Collective communications are an indispensable part of distributed train...
share
research
∙
03/24/2023
ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale
As deep learning models and input data are scaling at an unprecedented r...
research
∙
10/09/2021
Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Models
The continuous growth in both size and training data for modern Deep Neu...
research
∙
09/24/2021
Exploring Multi-dimensional Hierarchical Network Topologies for Efficient Distributed Training of Trillion Parameter DL Models
Deep Neural Networks have gained significant attraction due to their wid...
research
∙
03/18/2021
