Presto has become a popular tool for data scientists and engineers dealing with multiple query languages, siloed databases and different types of storage. Its high-performance capabilities enable users to query large volumes of data in real-time, regardless of where the data is located, using a simple ANSI SQL interface. Presto’s speed and performance at executing queries on large volumes of data have made it an indispensable tool for some of the largest companies in the world including Facebook, Airbnb, Netflix, Microsoft, Apple (iOS) and AWS (Athena and Amazon s3).

Presto architecture is unique in that it is built to query data no matter where the data is being stored, making it more scalable and efficient than other, similar solutions. Presto queries allow engineers to use data without having to physically move it from location to location. This is an important capability to have as organizations deal with an ever-increasing amount of data they need to store and analyze.

Presto was built to empower data scientists and engineers to interactively query vast amounts of data regardless of the source or type of storage. Because Presto doesn’t store data, but rather communicates with a separate database for its queries, it is more flexible than its competitors and can scale queries up or down swiftly based on the shifting needs of the organization. According to an IBM whitepaper, Presto, optimized for business intelligence (BI) workloads, can help enterprises optimize the pricing of their data warehouses and reduce costs by up to 50 percent. 

Here are some of the key benefits to using a Presto workflow:

Lower costs: As the size of data warehouses and the number of users conducting queries grows, it’s not uncommon for enterprises to see their costs rapidly increase. Presto, however is optimized for large amounts of small queries, making it easy to query any amount of data while also keeping costs down. Also, since Presto is open-source, there are no fees associated with deploying it, which can result in significant savings for enterprises looking to process large volumes of data.

Increased scalability: It’s common for engineers to set up multiple engines and languages on a single data lake storage system, which can make it necessary to re-platform in the future and limit the scalability of their solution. With Presto, all queries are conducted using the universal ANSI SQL language and interface, making re-platforming redundant. Additionally, Presto can be used for both small and large amounts of data and easily scaled up from one or two users to thousands. Presto deploys multiple compute engines with unique SQL dialects and APIs, making it an ideal tool for scaling workloads that could be too complex and time-intensive for teams of engineers and data scientists to handle.

Better performance: While many query engines that run SQL on Hadoop are restricted in their compute performance because they are built to write their results to disk, Presto’s distributed in-memory model enables it to run large amounts of interactive queries at once against large data sets. Following a classic massively parallel processing (MPP) design, Presto schedules as many queries as it can on a single worker node and uses in-memory streaming shuffle to increase its processing speeds even more. Executing tasks in-memory makes writing and reading from the disk between stages redundant and shortens the time of each query execution, making Presto a lower latency option than its competitors.

Improved flexibility: Presto uses a plug-and-play model for all its data sources including Cassandra, Kafka, MySQL, Hadoop distributed file system (HDFS), PostgreSQL and others, making querying across them faster and easier than with other comparable tools that lack this functionality. Also, Presto’s flexible architecture means it isn’t restricted to a single vendor but runs on most Hadoop distributions, making it one of the most portable tools available.

While Presto isn’t the only SQL-on-Hadoop option available to developers and data engineers, its unique architecture that keeps query functionality separate from data storage makes it one of the most flexible. Unlike other tools, Presto siloes off the query engine from the data storage and uses connectors to communicate between them. This added functionality gives engineers more flexibility than other tools in how they construct solutions using Presto.

Presto uses an MPP database management system with one coordinator node that works in tandem with other nodes. A Presto ecosystem is made up of three server types, a coordinator server, worker server and resource manager server.

Coordinator: A coordinator is considered the “brains” of a Presto installation. It is responsible for some of the most critical tasks including parsing statements, planning queries and managing Presto worker nodes. Ultimately, it is responsible for retrieving data from the worker nodes and delivering the results to the client.

Worker: The worker is responsible for gathering data from the worker nodes and ensuring the smooth exchange of data between itself and the connectors.

Resource manager: The resource manager gathers data from all the coordinator and worker nodes and creates a global view, or a “Presto cluster”. 

When the Presto coordinator SQL server receives an SQL query from a user, the first thing it does is use a custom query to parse, plan and schedule a distributed plan across the other nodes. Presto Rest API is used to submit query statements for execution on a server and to retrieve the results for the client. Presto supports standard ANSI SQL meanings, including joins, queries, sub-queries and aggregations. Once it has compiled the query, Presto parses the request into different stages between the worker nodes.

Since Presto was built on the concept of data abstraction, it is extensible to any data source and can easily query data sources such as data lakes, data warehouses and relational databases. Data abstraction is a programming process that allows data to be stored and manipulated more efficiently by separating its representation from its physical storage. This abstraction allows for the query engine to focus exclusively on the aspects of the data that are relevant to its query. Using the process of data abstraction, data is queried wherever it is being stored rather than once it has been moved into another analytics system.

A brief history of Presto

Initially developed at Facebook to run interactive queries on a massive Apache Hadoop data warehouse, Presto’s developers always envisioned it as open-source software and sought to make it free for commercial use so anyone could use it for data analytics and data management. In 2013, it was open-sourced on GitHub for anyone to download under the Apache Software license. In 2019, three of the original members of the Presto development team left the project and founded a “fork” of Presto known as the Presto Software Foundation, or more commonly, as prestosql.

The Linux foundation and other open-source communities offer webinars and training on Presto in English and other languages for engineers and developers looking to obtain certification. These forums are also a good place to visit to find out what’s new in Presto.