In Hive on MR3 on Kubernetes, a DAGAppMaster runs in one of the following three modes:
- LocalThread mode: A DAGAppMaster starts as a new thread inside the MR3Client (typically HiveServer2).
- LocalProcess mode: A DAGAppMaster starts as a new process on the same Pod where the MR3Client is running.
- Kubernetes mode: A DAGAppMaster starts on its own Pod.
A ContainerWorker runs in one of the following two modes:
- Local mode: A ContainerWorker starts as a thread inside the DAGAppMaster.
- Kubernetes mode: A ContainerWorker starts as a Pod in a Kubernetes cluster.
Hive on MR3 support all combinations of these modes. Here are schematic descriptions of the six combinations. Note that in the current implementation, HiveServer2 always starts in a Pod.
-
DAGAppMaster in LocalThread mode, ContainerWorker in Local mode
-
DAGAppMaster in LocalThread mode, ContainerWorker in Kubernetes mode
-
DAGAppMaster in LocalProcess mode, ContainerWorker in Local mode
-
DAGAppMaster in LocalProcess mode, ContainerWorker in Kubernetes mode
-
DAGAppMaster in Kubernetes mode, ContainerWorker in Local mode
-
DAGAppMaster in Kubernetes mode, ContainerWorker in Kubernetes mode
A DAGAppMaster mode can be specified with key mr3.master.mode
in mr3-site.xml
:
- LocalThread mode:
mr3.master.mode=local-thread
- LocalProcess mode:
mr3.master.mode=local-process
- Kubernetes mode:
mr3.master.mode=kubernetes
A ContainerWorker mode can be specified with key mr3.am.worker.mode
in mr3-site.xml
:
- Local mode:
mr3.am.worker.mode=local
- Kubernetes mode:
mr3.am.worker.mode=kubernetes