MOVie: GPU Memory Optimization for Large-Scale DNNs Training with Virtual Memory Management

As the complexity of models and the scale of parameters grows rapidly, the limited memory capacity of intelligent acceleration devices such as GPUs has become a constraint on the development of large-scale deep neural networks (DNNs). Deep learning frameworks employ caching allocators to manage memory pools through a "split-merge" mechanism, enhancing GPU memory utilization while inevitably introducing memory fragmentation. While common memory optimization techniques effectively alleviate memory pressure during the training phase of DNNs, they also exacerbate the fragmentation issue. Existing methods rely on pre-acquired runtime information to guide memory allocation, which fails to adapt to dynamic and complex large-scale DNNs training scenarios. This paper proposes MOVie, a GPU memory optimization method for large-scale DNNs training based on virtual memory management. MOVie utilizes an over-subscribed virtual memory merging mechanism and a dynamic programming algorithm to combine non-contiguous memory blocks. Experimental results demonstrate that for various large-scale DNNs, MOVie can improve GPU memory utilization by up to 21.76% and achieves an average performance improvement of 1.5 times compared to the state-of-the-art methods.