Techniques for Moment Retrievals and Filtering from Large Volumes of Multimodal Data

This paper focuses on applications of parallel processing and machine learning techniques for the retrieval and filtering of selected “moments” from textual and video-based multimodal data. Given a set of textual and video criteria provided by the user, the system extracts moments (sequences of frames) of a fixed duration from a large video file and stores them. An approach that supports the parallelization of this filtration process is presented. Experiments with Apache Spark and TensorFlow-based prototypes developed to demonstrate the efficacy of the proposed approach were conducted for various combinations of system and workload parameters. Insights into system performance including the reduction in processing time that accrues from applying the proposed parallel processing techniques for data filtering are reported in the paper.