Rise the Momentum: A Method for Reducing the Training Error on Multiple GPUs
Rise the Momentum: A Method for Reducing the Training Error on Multiple GPUs
Yu Tang,Lujia Yin,Zhaoning Zhang,Dongsheng Li
2019 · DOI: 10.1007/978-3-030-38961-1_4
International Conference on Algorithms and Architectures for Parallel Processing · 1 Citations
TLDR
Control the total batchsize and lowering batchsize on each GPU by increasing the number of GPUs in distributed training to improve training performance on condition of multiple GPUs of constant large batchsize.