Efficiency Analysis of the access method with the cascading Bloom filter to the data warehouse on the parallel computing platform
Efficiency Analysis of the access method with the cascading Bloom filter to the data warehouse on the parallel computing platform
Y. Grigoriev,V. Proletarskaya,E. Ermakov,O. Ermakov
TLDR
A new method was developed with a cascading Bloom filter for executing SQL queries in the Apache Spark parallel computing environment and full-scale experiments were carried out, which confirmed the effectiveness of the developed method.
Abstract
A new method was developed with a cascading Bloom filter (CBF) for executing SQL queries in the Apache Spark parallel computing environment. It includes the representation of the original query in the form of several subqueries, the development of a connection graph and the transformation of subqueries, the definition of connections where it is necessary to use Bloom filters, the representation of the graph in terms of Spark. On the example of the query Q3 of the TPC-H test, full-scale experiments were carried out, which confirmed the effectiveness of the developed method.