The user can use Kerberos on Amazon EKS in the same way as on Kubernetes in general. Assuming that a KDC (Key Distribution Center for Kerberos tickets) is already set up, we show those configurations relevant to using Kerberos.

Configuring for using Kerberos

kubernetes/conf/krb5.conf

Provide the realm and the address of the KDC.

$ vi kubernetes/conf/krb5.conf

[libdefaults]
  dns_lookup_realm = false
  ticket_lifetime = 24h
  renew_lifetime = 7d
  forwardable = true
  rdns = false
  default_realm = PL
  default_ccache_name = /tmp/krb5cc_%{uid}

[realms]
  PL = {
    admin_server = your.kdc.address
    kdc = your.kdc.address
  }

kubernetes/env.sh

Set the following environment variables appropriately:

$ vi kubernetes/env.sh

CREATE_KEYTAB_SECRET=true
CREATE_WORKER_SECRET=true

METASTORE_SECURE_MODE=true
HIVE_METASTORE_KERBEROS_PRINCIPAL=hive/admin@PL
HIVE_METASTORE_KERBEROS_KEYTAB=$KEYTAB_MOUNT_DIR/hive-admin.keytab

HIVE_SERVER2_AUTHENTICATION=KERBEROS
HIVE_SERVER2_KERBEROS_PRINCIPAL=hive/admin@PL
HIVE_SERVER2_KERBEROS_KEYTAB=$KEYTAB_MOUNT_DIR/hive-admin.keytab

TOKEN_RENEWAL_HDFS_ENABLED=false
  • Set TOKEN_RENEWAL_HDFS_ENABLED to false unless the data source includes HDFS..
  • The service keytab file(s) should be copied to the directory kubernetes/key by the user.

kubernetes/conf/core-site.xml

$ vi kubernetes/conf/core-site.xml

<property>
  <name>hadoop.security.authentication</name>
  <value>kerberos</value>
</property>

<property>
  <name>ipc.client.fallback-to-simple-auth-allowed</name>
  <value>true</value>
</property>
  • Set the configuration key hadoop.security.authentication to kerberos to enable Kerberos authentication.

kubernetes/conf/hive-site.xml

$ vi kubernetes/conf/hive-site.xml

<property>
  <name>hive.mr3.dag.additional.credentials.source</name>
  <value></value>
</property>
  • Set the configuration key hive.mr3.dag.additional.credentials.source to empty because we do not need additional credentials in order to read S3.

env.sh for running Beeline using hive/run-beeline.sh in the MR3 release

The user should set the following environment variables appropriately:

$ vi env.sh

HIVE3_SERVER2_HOST=15.164.117.52

HIVE_SERVER2_AUTHENTICATION=KERBEROS
HIVE_SERVER2_KERBEROS_PRINCIPAL=hive/admin@PL
HIVE_SERVER2_KERBEROS_KEYTAB=/home/gla/mr3-run/tools/krb5-kdc/hive-admin.keytab

Troubleshooting

If the KDC is not set properly, Metastore may fail with the following error:

Exception in thread "main" org.apache.hadoop.security.KerberosAuthException: failure to login: for principal: hive/admin@PL from keytab /opt/mr3-run/key/hive-admin.keytab javax.security.auth.login.LoginException: ICMP Port Unreachable

This error usually occurs when Metastore cannot reach the KDC via ports 88 and 749. In particular, make sure that the KDC is reachable via UDP ports 88 and 749 as well as TCP ports 88 and 749.

Calendar Feb 6, 2023