基于ambari 2.6.2.0的docker镜像安装大数据服务组件

之前有使用docker搭建了一套Apache Hadoop版本的大数据平台，参考我的码云地址：https://gitee.com/elbertmalone/HadoopHACluster，整个编写脚本和搭建过程花了很多时间，且灵活性不好。Ambari在大数据集群部署方面有得天独厚的优势，但是集群操作系统安装准备工作以及基础包的安装还是需要花费很多的时间。为了节省大数据集群的部署时间接下来我们用Docker容器化的方案部署Ambari。

费话少说，放码出来。让我们开始吧！

Ambari的架构

从Ambari的架构主要有两个组件：Ambari Server和Ambari Agent。

Ambari架构图

编写CentOS7支持Systemd镜像Dockfile

FROM centos:7

MAINTAINER elbert.lau

# 2, Setting up systemd
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;

VOLUME [ "/sys/fs/cgroup" ]
ENTRYPOINT ["/usr/sbin/init"]

RUN yum install -y systemd* && yum clean all

构建ambari-systemd镜像

#!/bin/bash

docker rmi local/ambari-systemd:2.6.2.0-155
docker build --rm -t local/ambari-systemd:2.6.2.0-155 .

编写Ambari Repos镜像Dockerfile

FROM local/ambari-systemd:2.6.2.0-155

MAINTAINER elbert.lau

RUN yum -y install httpd; yum clean all; systemctl enable httpd.service

ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari-2.6.2.0-centos7.tar.gz /var/www/html

ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz /var/www/html

ADD http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz /var/www/html/HDP-UTILS

ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/HDP-GPL-2.6.5.0-centos7-gpl.tar.gz /var/www/html

EXPOSE 80

构建ambari-repos镜像

#!/bin/bash

docker rmi local/ambari-repos:2.6.2.0-155
docker build --rm -t local/ambari-repos:2.6.2.0-155 .

编写Ambari Base镜像Dockerfile

FROM local/ambari-systemd:2.6.2.0-155

MAINTAINER elbert.lau

# 1, epel-release install
RUN yum install epel-release -y && yum clean all
RUN yum update -y && yum clean all

RUN yum install -y yum-utils yum-plugin-ovl tar git curl bind-utils unzip wget && yum clean all

# 2, Setup sshd
RUN yum install -y openssh-server openssh-clients && yum clean all
RUN systemctl enable sshd

# 3, kerberos client
RUN yum install -y krb5-workstation && yum clean all

# 4, initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start postgresql
RUN yum install -y initscripts && yum clean all

RUN curl -o /usr/bin/jq http://stedolan.github.io/jq/download/linux64/jq && chmod +x /usr/bin/jq

ENV JDK_ARTIFACT jdk-8u112-linux-x64.tar.gz
ENV JDK_VERSION jdk1.8.0_112

# 5, install Ambari specified 1.8 jdk
RUN mkdir /usr/jdk64 && cd /usr/jdk64 && wget http://public-repo-1.hortonworks.com/ARTIFACTS/$JDK_ARTIFACT \
&& tar -xf $JDK_ARTIFACT && rm -f $JDK_ARTIFACT

ENV JAVA_HOME /usr/jdk64/$JDK_VERSION
ENV PATH $PATH:$JAVA_HOME/bin

# 6, jce
ADD http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip $JAVA_HOME/jre/lib/security/
RUN cd $JAVA_HOME/jre/lib/security && unzip jce_policy-8.zip \
&& rm -f jce_policy-8.zip && mv UnlimitedJCEPolicyJDK8/*jar . && rm -rf UnlimitedJCEPolicyJDK8

ADD etc/yum.conf /etc/yum.conf

# 7, for ERROR: SSLError: Failed to connect. Please check openssl library versions.
RUN sed -i 's/verify=platform_default/verify=disable/' /etc/python/cert-verification.cfg

构建ambari-base镜像

#!/bin/bash

docker rmi local/ambari-base:2.6.2.0-155
docker build --rm -t local/ambari-base:2.6.2.0-155 .

编写Ambari Server镜像Dockerfile

FROM local/ambari-base:2.6.2.0-155

MAINTAINER elbert.lau

# 1, add ambari repo
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo /etc/yum.repos.d/ambari.repo
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo /etc/yum.repos.d/hdp.repo
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo /etc/yum.repos.d/hdp.gpl.repo

# 2, initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start postgresql
RUN yum -y install ambari-server; yum clean all

# 3, add psql connectors to ambari-server so it can be downloaded by services (e.g.: Ranger)
ADD https://jdbc.postgresql.org/download/postgresql-42.2.5.jar /var/lib/ambari-server/resources/postgres-jdbc-driver.jar

RUN echo DefaultEnvironment=\"JAVA_HOME=$JAVA_HOME\" >> /etc/systemd/system.conf

# 4, ambari server setup
RUN ambari-server setup --silent --java-home $JAVA_HOME; systemctl enable ambari-server.service

EXPOSE 8080

构建ambari-server镜像

#!/bin/bash

docker rmi local/ambari-server:2.6.2.0-155
docker build --rm -t local/ambari-server:2.6.2.0-155 .

编写Ambari Agent镜像Dockerfile

FROM local/ambari-base:2.6.2.0-155

MAINTAINER elbert.lau

# 1, add ambari repo
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo /etc/yum.repos.d/ambari.repo
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo /etc/yum.repos.d/hdp.repo
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo /etc/yum.repos.d/hdp.gpl.repo

# 2, install ambari agent
RUN yum -y install ambari-agent; yum clean all

RUN echo DefaultEnvironment=\"JAVA_HOME=$JAVA_HOME\" >> /etc/systemd/system.conf

构建ambari-agent镜像

#!/bin/bash

docker rmi local/ambari-agent:2.6.2.0-155
docker build --rm -t local/ambari-agent:2.6.2.0-155 .

参考资料

【1】https://github.com/sequenceiq/docker-ambari