Research on Query Task Fragmentation in the Scenario of Storage and Compute Separation

The cloud-native database is one of the hottest topics in database research. Storage-compute separation architecture with cloud characteristics is designed to process complex and changeable workloads to reduce resource coupling for cloud-native databases. During processing query loads, the computing layer pushes the operator tasks in execution plans generated from loads down to the storage layer through the network. Query optimization techniques such as operator pushdown and parallel computing have been developed and perfected to improve query speed in storage and compute separation architecture. But at the same time, these technologies also increase the complexity of execution plans, leading to an increase in tasks. Moreover, the experiment shows that under the same query load, the cloud-native database executes a significantly greater number of tasks compared to the distributed database. Given the above phenomenon, the concept of query task fragmentation is proposed. From the view of operator pushdown and parallel computing query optimization, the presence of query task fragmentation is verified through experiments. Experimental results based on the TPC-H benchmark demonstrate that, compared to MySQL distributed cluster, the average number of operator tasks in the execution plan, generated through operator pushdown and parallel computing for the cloud-native databases, increases by 20.17% and 39.49%, respectively. Finally, the impact of task fragmentation is analyzed from the perspectives of technical research and the application of cloud-native databases.