MR3
- Support DAG priority schemes (specified by
mr3.dag.priority.scheme)
and Vertex priority schemes (specified by mr3.vertex.priority.scheme).
- Support secure shuffle (using SSL mode) without requiring separate configuration files.
- ContainerWorker tries to avoid OutOfMemoryErrors by sleeping after a TaskAttempt fails (specified by
mr3.container.task.failure.num.sleeps).
- Errors from InputInitializers are properly passed to MR3Client.
- MasterControl supports two new commands for gracefully stopping DAGAppMaster and ContainerWorkers.
Hive on MR3
- Allow fractions for CPU cores (with
hive.mr3.resource.vcores.divisor).
- Support rolling updates.
- Hive on MR3 can access S3 using AWS credentials (with or without Helm).
- On Amazon EKS, the user can use S3 instead of PersistentVolumes on EFS.
- Hive on MR3 can use environment variables
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to access S3 outside Amazon AWS.