Skip to main content

Release 2.3 (2025-12-4)

MR3

Bug fixes

  • Fix a bug where DAGAppMaster schedules Tasks of the first DAG only one at a time in each ContainerWorker, thereby delaying its execution.
  • Fix a concurrency bug that stalls the execution of Tasks when all fetchers fail.
  • Fix a bug that sometimes generates duplicate output data during pipelined shuffling.

Backpressure

  • Extend the backpressure mechanism so that a ContainerWorker can request all ShuffleServers to stop sending shuffle requests. A new configuration key tez.shuffle.max.block.requests.thread.multiple specifies the threshold for the total number of active shuffle requests that triggers backpressure.

Shuffling

  • A LogicalInput can use a multiple of memory allocated to each Task for storing shuffle input data. A new configuration key tez.runtime.free.memory.factor.for.fetched.input specifies a multiplier used in calculating the total amount of free memory for storing shuffle input data.
  • Implement streaming mode for unordered shuffle, which completely eliminates writes to local disks. Set the new configuration key tez.runtime.shuffle.unordered.memory.streaming to true to enable streaming mode for unordered shuffle.

MR3-UI

  • MR3-UI can display the details of each operator.

Hive on MR3

  • The configuration key hive.mr3.am.task.concurrent.run.threshold.percent uses floating point numbers, not integers.