Hive on MR3

Using executable scripts:

Hive on MR3, Spark on MR3, Ranger, Grafana, MR3-UI, Superset

Using TypeScript code:

  • On EKS with Autoscaling shows how to opeate Hive/Spark on MR3 on Amazon EKS along with Ranger, Grafana, MR3-UI, and Superset, where Hive/Spark on MR3 is configured to use autoscaling.

mr3-typescript-eks-components.png

  • The user can connect to a public HiveServer2 (via JDBC/ODBC) which is exposed to the outside of the Kubernetes cluster.
  • The user can connect to Apache HTTP Server which serves as a gateway to Superset, Ranger, Grafana, MR3-UI, and Spark UI.
  • Superset connects to an internal HiveServer2 which is not exposed to the outside of the Kubernetes cluster. Superset with an internal HiveServer2 is optional.
  • All HiveServer2 instances share a common MR3 DAGAppMaster and its ContainerWorkers. Autoscaling is enabled by default.
  • Multiple Spark drivers sharing a common MR3 DAGAppMaster and its ContainerWorkers can run inside the Kubernetes cluster. Autoscaling is enabled by default.
  • HiveServer2 and Spark drivers are automatically configured to share Metastore.
Calendar Dec 6, 2023