DISTMM: Accelerating Distributed Multimodal Model Training

A new pipeline execution primitive, called batch-sync instruction, and a corresponding schedule, called D IST MM-Pipe are proposed, which addresses the limitation of existing pipeline execution schedules for multimodal training with contrastive loss.