Quickstep: A Data Platform Based on the Scaling-In Approach

Abstract

Modern servers pack enough storage and computing power that just a decade ago was spread across a modest- sized cluster. This paper presents a prototype system, called Quickstep, to exploit the large amount of paral- lelism that is packed inside such modern servers. Quick- step builds on a vast body of previous work on meth- ods for organizing data, optimizing, scheduling and ex- ecuting queries, and brings them together in a single sys- tem. Quickstep also includes new query processing meth- ods that go beyond previous approaches. To keep the project focused, the project’s initial target is read-mostly in-memory data warehousing workloads in single-node settings. In this paper, we describe the design and imple- mentation of Quickstep for this target application space. In this paper, we also present experimental results com- paring the performance of Quickstep to a number of other systems. These experiments show that Quickstep is of- ten faster than many other contemporary systems, and in some cases faster by an order-of-magnitude. Quickstep is an Apache (incubating) project and lives at: https:// github.com/apache/incubator-quickstep.