This page shows how to run Hive 3 on MR3 in a single Docker container using a pre-built all-in-one Docker image mr3project/hivemr3-all:1.8 available at DockerHub.

By following the instruction, the user will learn:

  1. how to run Hive on MR3 in a single Docker container
  2. how to create Beeline connections and send queries to HiveServer2
  3. how to change configurations of HiveServer2

For running Beeline outside the container, we assume that a client program to connect to HiveServer2 is already installed. This scenario should take less than 30 minutes to complete.

For asking any questions, please visit MR3 Google Group or join MR3 Slack.

Starting the all-in-one container

In order to start an all-in-one container, execute the following command in which we specify mr3project/hivemr3-all:1.8 as the name of the Docker image.

$ sudo docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 9852:9852 mr3project/hivemr3-all:1.8

The initialization script installs MySQL 5.6 and downloads mysql-connector-java-8.0.17.tar.gz for a MySQL connector.

It uses the following ports for HiveServer2:

  • 9852 for HiveServer2

The user can inspect the log file run.log inside the container to check if MetaStore and HiveServer2 have started properly.

$ docker exec -it fdf73c08660a6d3b846f7d841eaba57d85d2b597a5112adb984ae9477331029b /bin/bash
[root@fdf73c08660a mr3-run]# tail -f run.log 
starting Metastore and HiveServer2

Running Beeline inside the container as user hive

Inside the container, download a sample dataset and execute /opt/mr3-run/hive/ as user hive.

$ docker exec -it fdf73c08660a6d3b846f7d841eaba57d85d2b597a5112adb984ae9477331029b /bin/bash
[root@fdf73c08660a mr3-run]# su hive
bash-4.2$ pwd
bash-4.2$ wget

bash-4.2$ hive/
Connecting to jdbc:hive2://fdf73c08660a:9852/;;;
Connected to: Apache Hive (version 3.1.3)
Driver: Hive JDBC (version 3.1.3)
Beeline version 3.1.3 by Apache Hive

Create a table called pokemon and import the sample dataset.

0: jdbc:hive2://fdf73c08660a:9852/> CREATE TABLE pokemon (Number Int,Name String,Type1 String,Type2 String,Total Int,HP Int,Attack Int,Defense Int,Sp_Atk Int,Sp_Def Int,Speed Int) row format delimited fields terminated BY ',' lines terminated BY '\n' tblproperties("skip.header.line.count"="1");

0: jdbc:hive2://fdf73c08660a:9852/> load data local inpath '/opt/mr3-run/pokemon.csv' INTO table pokemon;

Execute queries.

0: jdbc:hive2://fdf73c08660a:9852/> select avg(HP) from pokemon;

0: jdbc:hive2://fdf73c08660a:9852/> create table pokemon1 as select *, IF(HP>160.0,'strong',IF(HP>140.0,'moderate','weak')) AS power_rate from pokemon;

0: jdbc:hive2://fdf73c08660a:9852/> select COUNT(name), power_rate from pokemon1 group by power_rate;
| _c0  | power_rate  |
| 363  | strong      |
| 336  | weak        |
| 108  | moderate    |
3 rows selected (1.265 seconds)

The user can find the warehouse directory /opt/mr3-run/work-dir/warehouse/.

bash-4.2$ ls work-dir/warehouse/
pokemon  pokemon1

Running Beeline outside the container as another user

The user may use any client program to connect to HiveServer2. In our example, we use the script included in the MR3 release (hive/

Open and set environment variables HIVE3_SERVER2_HOST, HIVE3_SERVER2_PORT, SECURE_MODE, and HIVE_SERVER2_AUTHENTICATION. Note that we do not use Kerberos for connecting to HiveServer2.


Run Beeline using --tpcds and --hivesrc3 options.

$ ./hive/ --tpcds --hivesrc3
beeline -u "jdbc:hive2://your.server.address:9852/;;" -n gla -p gla --hiveconf hive.querylog.location=/data1/gla/mr3-run-1.0/hive/run-beeline-result/hive-mr3-5ba3d48-2020-10-28-17-02-06-d0064f30/hive-logs
Connecting to jdbc:hive2://your.server.address:9852/;;
Connected to: Apache Hive (version 3.1.3)
Driver: Hive JDBC (version 3.1.3)
Beeline version 3.1.3 by Apache Hive
0: jdbc:hive2://your.server.address:9852/>

Note that in our example, we connect to HiveServer2 as user gla, not hive.

Now execute a sample query.

0: jdbc:hive2://your.server.address:9852/> use default;
0: jdbc:hive2://your.server.address:9852/> select count(*) from pokemon;

Changing configurations for HiveServer2

In order to change configurations for HiveServer2, start an all-in-one container with an initial command /usr/sbin/init.

$ sudo docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 6080:6080 -p 8188:8188 -p 9852:9852 mr3project/hivemr3-all:1.8 /usr/sbin/init

Then update /opt/mr3-run/ and configuration files under the directory /opt/mr3-run/conf as necessary.

$ docker exec -it c85a7579e5092e4cdfe12b1dfdc5d78178c76f2b92d4d5657cacdf911962f83b /bin/bash
[root@c85a7579e509 mr3-run]# vi
[root@c85a7579e509 mr3-run]# vi conf/core-site.xml
[root@c85a7579e509 mr3-run]# vi conf/mr3-site.xml
[root@c85a7579e509 mr3-run]# vi conf/hive-site.xml

Finally run the initialization script

[root@c85a7579e509 mr3-run]# ./