Skip to main content

Release 2.4 (2026-2-8)

MR3​

Bug fixes​

  • Fix a bug that DAGs sometimes get stuck during Vertex rerun after ContainerWorkers are killed.

Shuffling​

  • ContainerWorkers continue to run after Fetcher threads fail with errors.
  • Fault tolerance is much faster when ContainerWorkers are killed.
  • Remove the configuration key tez.runtime.shuffle.max.input.hostports.

Task scheduling​

  • Introduce a new configuration key mr3.taskattempt.queue.scheme.strict to specify that TaskScheduler should strictly respect location hints of Tasks.

DAG scheduling​

  • (Experimental) Introduce a new scheme fair for DAG scheduling. Set the configuration key mr3.dag.queue.scheme to fair to try the new scheme. The scheme fair uses three auxiliary configuration keys: mr3.dag.queue.fair.miss.host.delay, mr3.dag.queue.fair.num.recent.dags, and mr3.dag.queue.fair.extra.worker.max.percent. This is an experimental feature (which is particularly effective in large clusters) and can be removed in future releases.