Unity: Accelerating DNN Training Through Joint Optimization of Algebraic Transformations and Parallelization
Unity: Accelerating DNN Training Through Joint Optimization of Algebraic Transformations and Parallelization
Colin Unger,Zhihao Jia,12 Authors,Alexander Aiken
2022 · DBLP: conf/osdi/UngerJ0LBNRPMML22
USENIX Symposium on Operating Systems Design and Implementation · 90 Citations
TLDR
Unity is presented, the first system that jointly optimizes algebraic transformations and parallelization in distributed DNN training, and is capable of integrating new DNN operators, parallelization strategies, and model architectures with minimal manual effort.