MaaI: ML-as-an-Infrastructure for Diverse Personalized Inference With Edge Offloading

The rapid advancements in large language models (LLMs) have transformed applications across industries, from natural language understanding to content generation. However, their escalating computational demands and dependence on cloud services create barriers to the widespread adoption of customized models. We introduce MaaI, an edge-offloading architecture that optimizes LLM deployment by distributing computational tasks and customization across edge devices and cloud servers. Crucially, MaaI identifies the optimal partition point to divide the model between the server and edge, ensuring that foundational components are allocated to edge devices while intensive computations remain in the cloud. Utilizing quantization and LoRA adapters, this approach enables users to leverage larger models with enhanced performance while minimizing server load. Furthermore, by retaining only the foundational model on the server and offloading customized adaptations to edge devices, MaaI facilitates scalable, personalized deployments without significantly increasing server resource requirements. Experimental results across diverse benchmarks and hardware configurations confirm MaaI’s effectiveness in optimizing performance, memory efficiency, and latency.