In Hive on MR3 on Kubernetes, a DAGAppMaster runs in one of the following three modes:
- LocalThread mode: A DAGAppMaster starts as a new thread inside the MR3Client (typically HiveServer2).
- LocalProcess mode: A DAGAppMaster starts as a new process on the same Pod where the MR3Client is running.
- Kubernetes mode: A DAGAppMaster starts on its own Pod.
A ContainerWorker runs in one of the following two modes:
- Local mode: A ContainerWorker starts as a thread inside the DAGAppMaster.
- Kubernetes mode: A ContainerWorker starts as a Pod in a Kubernetes cluster.
Combinations of DAGAppMaster and ContainerWorker modes
Hive on MR3 support all combinations of these modes. Here are schematic descriptions of the six combinations. Note that in the current implementation, HiveServer2 always starts in a Pod.
-
DAGAppMaster in LocalThread mode, ContainerWorker in Local mode
-
DAGAppMaster in LocalThread mode, ContainerWorker in Kubernetes mode
-
DAGAppMaster in LocalProcess mode, ContainerWorker in Local mode
-
DAGAppMaster in LocalProcess mode, ContainerWorker in Kubernetes mode
-
DAGAppMaster in Kubernetes mode, ContainerWorker in Local mode
-
DAGAppMaster in Kubernetes mode, ContainerWorker in Kubernetes mode
Specifying DAGAppMaster and ContainerWorker modes
A DAGAppMaster mode can be specified with key mr3.master.mode in mr3-site.xml:
- LocalThread mode:
mr3.master.mode=local-thread - LocalProcess mode:
mr3.master.mode=local-process - Kubernetes mode:
mr3.master.mode=kubernetes
A ContainerWorker mode can be specified with key mr3.am.worker.mode in mr3-site.xml:
- Local mode:
mr3.am.worker.mode=local - Kubernetes mode:
mr3.am.worker.mode=kubernetes
