Skip to main content

TPC-DS Benchmark: Trino 476, Spark 4.0.0, and Hive 4 on MR3 2.1 (MPP vs MapReduce)

· 9 min read
Sungwoo Park
MR3 Architect and Developer

In our previous article, we evaluated the performance of Trino 468, Spark 4.0.0-RC2, and Hive 4.0.0 on MR3 2.0 using the TPC-DS Benchmark with a scale factor of 10TB.

  • Correctness. Trino returns incorrect results for both subqueries of query 23.
  • Total execution time (Sequential). Trino is the fastest, followed closely by Hive on MR3 (4,442 seconds vs 4,874 seconds). Spark is the slowest, skewed by a few outlier queries (15,678 seconds).
  • Average response time (Sequential). Trino maintains the lead in average response time, with Hive on MR3 again closely behind (17.49 seconds vs 19.76 seconds).
  • Longest execution time (Concurrent). Under concurrent workloads (10, 20, and 40 clients), Hive on MR3 consistently outperforms both Trino and Spark.

TPC-DS Benchmark: Trino 468, Spark 4.0.0-RC2, and Hive 4 on MR3 2.0

· 15 min read
Sungwoo Park
MR3 Architect and Developer

In this article, we evaluate the performance of Trino, Spark, Hive on Tez, and Hive on MR3 using the TPC-DS Benchmark with a scale factor of 10TB.

  1. Trino 468 (released in December 2024)
  2. Spark 4.0.0-RC2 (released in March 2025)
  3. Hive 4.0.0 on Tez (built in February 2025)
  4. Hive 4.0.0 on MR3 2.0 (released in April 2025)

Trino is an MPP-style query engine and is not fault-tolerant. The other three systems are fully fault-tolerant.

Optimizing Query Compilation in Hive 4 on MR3

· 7 min read
Sungwoo Park
MR3 Architect and Developer

Introduction

In our previous article, we evaluated the performance of Hive 4 on MR3 1.11 and Trino 453 on the 10TB TPC-DS benchmark. The results can be summarized as follows:

  • In terms of the total running time, Hive 4 on MR3 runs slightly faster than Trino -- Hive 4 on MR3 5744 seconds vs Trino 5798 seconds.
  • In terms of the geometric mean of running times, Trino responds about 15 percent faster than Hive 4 on MR3 -- Trino 17.99 seconds vs Hive 4 on MR3 21.02 seconds.

Performance Evaluation of Trino and Hive on MR3 using the TPC-DS Benchmark

· 5 min read
Sungwoo Park
MR3 Architect and Developer

Introduction

In our previous article, we evaluate the performance of Trino 418 and Hive on MR3 1.7 using the TPC-DS Benchmark with a scale factor of 10TB.

  • In terms of the total running time, the two systems are comparable: Trino 7424 seconds vs Hive on MR3 7415 seconds.
  • In terms of the geometric mean of running times, Trino is faster than Hive on MR3: Trino 21.75 seconds vs Hive on MR3 27.68 seconds.
  • Trino returns wrong answers on query 23 after running for 1756 seconds.
  • Trino fails to complete query 72 after running for 156 seconds.

Performance Tuning for Single-table Queries

· 5 min read
Sungwoo Park
MR3 Architect and Developer

Introduction

In our previous article, we have shown that Hive on MR3 1.7 runs much faster than Spark 3.4.0 on the TPC-DS benchmark with a scale factor of 10TB (7415 seconds vs 19669 seconds). The performance gap is expected to widen further due to improvements in Hive on MR3 1.8 (6867 seconds vs 7415 seconds). Still, however, there is a category of queries on which Hive on MR3 seems noticeably slower than Spark: single-table queries with no joins.

Spark on MR3 - A New Way to Run Apache Spark

· 10 min read
Sungwoo Park
MR3 Architect and Developer

Introduction

MR3 is a general purpose execution engine that provides native support for Hadoop and Kubernetes. While Hive on MR3 is its main application, MR3 can easily execute MapReduce/Tez jobs as well. For example, Hive on MR3 supports compaction without relying on MapReduce because MapReduce jobs performing compaction are directly sent to MR3.