In this section, we describe major features of MR3.
One can try these features by changing corresponding configurations in mr3-site.xml
in the classpath.
We assume some familiarity with such notions as DAG, Vertex, Task, and TaskAttempt.
For clarity, we refer to client program in MR3 as MR3Client, ApplicationMaster in MR3 as DAGAppMaster, and container in MR3 as ContainerWorker.
- MR3Client
- DAGAppMaster and ContainerWorker Modes
- ContainerGroup
- Concurrent DAGs
- DaemonTask
- Task Scheduling
- Fault Tolerance
- Node Blacklisting
- Autoscaling
- MR3 Shuffle Handler
- Managing Fetchers
- Speculative Execution
- Eliminating Fetch Delays
- Recycling ContainerWorkers
- Using Celeborn as Remote Shuffle Service