Model-Driven Right-Sizing of Offloading in Data Processing Pipelines

The performance of modern Big Data systems used for data processing is often bottlenecked by data movement across various components. Offloading part of the processing closer to the storage and network to reduce this bottleneck is a compelling idea, and in today’s hardware landscape, there are many different types of Smart Storage, Smart NIC, or Smart Switch devices one could choose from. One challenge, however, is that it is often unclear at design time what improvements of the end workload one can achieve with a given hardware. Co-design is typically mentioned in related work as the solution, but it is far from obvious what this entails in practice: what information about the hardware, software system, and workload is taken into account for decision-making is often implicit in related work. In this work, we propose a model-driven methodology for right-sizing offloading to benefit an end workload. Our methodology determines the target processing rate an offload device should have in a specific data processing system, without over-fitting. This helps designers pick the right hardware for offloading. Our methodology relies on modeling the system as a network of queues, based on different levels of information about the system, which allows determining the general usefulness of offload and the specific benefits to a workload of interest. We demonstrate how our methodology avoids under- or over-provisioning offload devices in a case study.