5 posts tagged with "LLAP"

Introduction​

In this article, we evaluate the performance of the following systems.

1. Spark 2.3.8
2. Spark 3.2.1
3. Hive 3.1.2 on MR3 1.4
4. Hive-LLAP in HDP 3.1.4 (3.1.0.3.1.4.0-315)

Introduction​

In our previous article published in October 2018, we use the TPC-DS benchmark to compare the performance of Hive-LLAP and SparkSQL 2.3.1 included in HDP 3.0.1 along with Hive 3.1.0 on MR3 0.4. In this article, we update the result by testing SparkSQL 2.3.2 included in HDP 3.1.4. As in the previous experiment, we use the TPC-DS benchmark.

Introduction​

We often ask questions on the performance of SQL-on-Hadoop systems:

• How fast is Hive-LLAP in comparison with Presto, SparkSQL, or Hive on Tez?
• As it is an MPP-style system, does Presto run the fastest if it successfully executes a query?
• As it stores intermediate data in memory, does SparkSQL run much faster than Hive on Tez in general?
• What is the best system for running concurrent queries?
• ...

NOTE: This article is superceded by a new article expanded with results of concurrency tests on newer versions of SQL-on-Hadoop systems.​

Introduction​

Hive running on top of MR3 0.2, or Hive-MR3 henceforth, supports LLAP (Low Latency Analytical Processing) I/O. In conjunction with the ability to execute multiple TaskAttempts concurrently inside a single ContainerWorker, the support for LLAP I/O makes Hive-MR3 functionally equivalent to Hive-LLAP. Hence Hive-MR3 can now serve as a substitute for Hive-LLAP in typical use cases.