Skip to main content

Release 3.0 (2026-8-2)

MR3​

DAG scheduling​

  • Introduce a new scheme fair for DAG scheduling. Set the configuration key mr3.dag.queue.scheme to fair to use the new scheme. The scheme fair uses the following auxiliary configuration keys:

    • mr3.dag.queue.fair.miss.host.delay
    • mr3.dag.queue.fair.num.recent.dags
    • mr3.dag.queue.fair.extra.worker.max.percent

Task scheduling​

  • Support the use of rack labels for scheduling hints with a new configuration key mr3.dag.taskattempt.queue.use.hint.rack.
  • Introduce a new configuration key mr3.taskattempt.queue.wait.per.vertex.cache to consider per-vertex cache.
  • Introduce a new configuration key mr3.dag.taskattempt.queue.match.host.first to schedule Tasks with matching location hints regardless of their priorities.

Shuffling​

  • Remove the configuration key tez.shuffle.vertex.manager.auto.parallelism.min.percent.
  • Implement a new strategy for the configuration key tez.shuffle-vertex-manager.use-stats-auto-parallelism in order to reduce the number of shuffle requests.

MR3-UI​

  • Implement TaskAttempt swimlanes.
  • The internal Timeline Server for MR3-UI attempts to recover from initialization failures.

Logging​

  • A log file is created for each DAG (e.g., dag_1783582797348_0112_44.log).

Hive on MR3​

  • Support Hive 4.2.0.
  • Optimize the performance of shuffle (by avoiding memory copies) and serialization/deserialization (by directly serializing/deserializing row batches).
  • HiveSplitGenerator does not use the configuration key dfs.blocksize in calculating the split size. It uses tez.grouping.min-size to calculate the minimum split size after considering mapreduce.input.fileinputformat.split.minsize.