MobiMind: A Lightweight and Fast Inference Framework of Deep Neural Networks for Mobile Devices

As intelligent application services transition from the cloud to mobile devices such as AIPC laptops and smart-phones, they face increasing computational and storage challenges. Running deep neural network (DNN) inference and real-time high-precision tasks on mobile platforms demands significant resources, pushing the limits of their capabilities. While extensive research on model compression and pruning techniques has shown promise, non-structured pruning maintains accuracy but lacks hardware efficiency, whereas structured pruning is hardware-friendly but often sacrifices accuracy. Additionally, efficient memory buffer utilization on mobile devices, particularly for AI inference and accelerator optimization, remains a persistent bottleneck. In this paper, We propose an optimized pruning method to enhance DNN inference on mobile devices, focusing on compiler-level optimization. This approach integrates techniques like Quantization and weight compression to improve hardware efficiency while maintaining accuracy. Using a recursive depth-first search strategy, it restricts exploration to canonical solutions, reducing the search space and enabling more effective pruning. Empirical results show up to an order-of-magnitude performance improvement The framework minimizes resource use, achieving real-time execution of demanding DNN models. It outperforms TensorFlow Lite and Tensor Virtual Machine(TVM) with speedups of up to 53.2× and 16.3×, respectively, without accuracy loss.