制作篇3 - 制作agent-server镜像包

最新推荐文章于 2024-05-09 16:48:31 发布

secretWHD

最新推荐文章于 2024-05-09 16:48:31 发布

阅读量3.6k

点赞数 2

分类专栏： CDH Docker 文章标签： big data 大数据 cloudera docker

本文链接：https://blog.csdn.net/secretWHD/article/details/120610708

版权

CDH 同时被 2 个专栏收录

10 篇文章 11 订阅

订阅专栏

Docker

7 篇文章 1 订阅

订阅专栏

该镜像包主要安装CDH 的daemons，agent服务

1. 创建并运行基础容器

启动基础容器，在此基础上安装agent-server所需的软件

docker run -d \
--hostname agentserver \
--name agentserver \
--privileged=true \
server/bashimage  \
/usr/sbin/init \
&& docker ps

结果：
9bba31143dcf70a4775a7d20085ea277f940f67e132cfe9b8d7001fa92189d87
CONTAINER ID   IMAGE              COMMAND            CREATED                  STATUS                  PORTS     NAMES
9bba31143dcf   server/bashimage   "/usr/sbin/init"   Less than a second ago   Up Less than a second   22/tcp    agentserver

2. 初始化容器环境

# 进入容器
docker exec -it --privileged=true agentserver /bin/bash 
 
# 初始化容器环境
ping www.baidu.com -c 3 \
&& yum update -y \
&& yum install -y openssh-clients kde-l10n-Chinese telnet reinstall glibc-common wget ntp net-tools firewalld httpd unzip \
&& yum clean all 
 
结果：
PING www.a.shifen.com (180.101.49.11) 56(84) bytes of data.
64 bytes from 180.101.49.11 (180.101.49.11): icmp_seq=1 ttl=127 time=54.7 ms
64 bytes from 180.101.49.11 (180.101.49.11): icmp_seq=2 ttl=127 time=65.2 ms
64 bytes from 180.101.49.11 (180.101.49.11): icmp_seq=3 ttl=127 time=87.8 ms...
Complete!
Loaded plugins: fastestmirror, ovl
Cleaning repos: base extras updates
Cleaning up list of fastest mirrors

3. 关闭防火墙

systemctl stop firewalld \
&& systemctl disable firewalld \
&& systemctl status firewalld 

结果：
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)

4. 配置中文环境

(
cat <<EOF
export LC_ALL=zh_CN.utf8
export LANG=zh_CN.utf8
export LANGUAGE=zh_CN.utf8
EOF
) >> ~/.bashrc \
&& localedef -c -f UTF-8 -i zh_CN zh_CN.utf8 \
&& source ~/.bashrc \
&& echo $LANG

结果：
zh_CN.utf8

5. 配置httpd ，新增yum源

此步骤是为了离线安装CDH的JDK，daemons ，agent

systemctl start httpd \
&& systemctl enable httpd \
&&(
cat <<EOF
[cloudera-manager]
name=Cloudera Manager 6.3.1
baseurl=http://172.16.0.2/cloudera-repos/
gpgcheck=0
enabled=1
EOF
) >> /etc/yum.repos.d/cloudera-manager.repo \
&& curl http://172.16.0.2/cloudera-repos/ \
&& yum clean all

结果：
<h1>Index of /cloudera-repos</h1>
  <table>
   <tr><th valign="top"><img src="/icons/blank.gif" alt="[ICO]"></th><th><a href="?C=N;O=D">Name</a></th><th><a href="?C=M;O=A">Last modified</a></th><th><a href="?C=S;O=A">Size</a></th><th><a href="?C=D;O=A">Description</a></th></tr>
   <tr><th colspan="5"><hr></th></tr>
<tr><td valign="top"><img src="/icons/back.gif" alt="[PARENTDIR]"></td><td><a href="/">Parent Directory</a>       </td><td>&nbsp;</td><td align="right">  - </td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/text.gif" alt="[TXT]"></td><td><a href="allkeys.asc">allkeys.asc</a>            </td><td align="right">2021-07-05 00:34  </td><td align="right"> 14K</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm">cloudera-manager-age..&gt;</a></td><td align="right">2021-06-15 16:00  </td><td align="right"> 10M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm">cloudera-manager-dae..&gt;</a></td><td align="right">2021-06-15 16:10  </td><td align="right">1.1G</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm">cloudera-manager-ser..&gt;</a></td><td align="right">2021-06-15 15:59  </td><td align="right"> 11K</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm">cloudera-manager-ser..&gt;</a></td><td align="right">2021-06-15 15:59  </td><td align="right"> 11K</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm">enterprise-debuginfo..&gt;</a></td><td align="right">2021-06-15 16:00  </td><td align="right"> 14M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="mysql-community-client-5.7.27-1.el7.x86_64.rpm">mysql-community-clie..&gt;</a></td><td align="right">2021-07-08 17:41  </td><td align="right"> 24M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="mysql-community-common-5.7.27-1.el7.x86_64.rpm">mysql-community-comm..&gt;</a></td><td align="right">2021-07-08 17:42  </td><td align="right">275K</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="mysql-community-devel-5.7.27-1.el7.x86_64.rpm">mysql-community-deve..&gt;</a></td><td align="right">2021-07-08 17:42  </td><td align="right">3.7M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="mysql-community-libs-5.7.27-1.el7.x86_64.rpm">mysql-community-libs..&gt;</a></td><td align="right">2021-07-08 17:42  </td><td align="right">2.2M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm">mysql-community-libs..&gt;</a></td><td align="right">2021-07-08 17:42  </td><td align="right">2.0M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="mysql-community-server-5.7.27-1.el7.x86_64.rpm">mysql-community-serv..&gt;</a></td><td align="right">2021-07-08 17:43  </td><td align="right">165M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/unknown.gif" alt="[   ]"></td><td><a href="oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm">oracle-j2sdk1.8-1.8...&gt;</a></td><td align="right">2021-06-15 16:05  </td><td align="right">176M</td><td>&nbsp;</td></tr>
<tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="repodata/">repodata/</a>              </td><td align="right">2021-09-12 16:35  </td><td align="right">  - </td><td>&nbsp;</td></tr>
   <tr><th colspan="5"><hr></th></tr>
</table>
</body></html>
...

6. 安装JDK，daemons ，agent

yum install -y oracle-j2sdk1.8 cloudera-manager-daemons cloudera-manager-agent \
&& rpm -qa | grep cloudera-manager \
&& (cat <<\EOF
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
export PATH=$PATH:$JAVA_HOME/bin
EOF
) >> ~/.bashrc \
&& source ~/.bashrc \
&& java -version

结果:
Complete!
cloudera-manager-agent-6.3.1-1466458.el7.x86_64
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
未出现上诉一模一样的结果，请重新执行一遍（cloudera-manager-daemons有点难装 ）

7. 修改agent配置

server_host 是CDH master-server的hostname，也就是master-server容器的主机名，根据现场情况修改

不然会导致CDH master-server找不到agent服务

sed -i 's/server_host=localhost/server_host=server001/g' /etc/cloudera-scm-agent/config.ini \
&& grep "server_host" /etc/cloudera-scm-agent/config.ini

结果：
server_host=server001

到这为止，agent-server容器已经安装了JDK，CDH的daemos，agent服务，只是安装好，并没有启动也无需启动。开始打成镜像包做备份

8. 将容器提交成镜像

接下来将agent-server容器打包成镜像。

该容器可以重复使用，CDH集群需要多个agent服务，启动相应数量的agent-server容器即可

# 停止asrserver00x容器,提交为镜像,查看镜像，打包镜像
docker stop agentserver \
    && docker commit agentserver agent-server/cdh:6.3.2 \
    && docker images \
    && docker save -o /root/agent-server.tar.gz agent-server/cdh:6.3.2 \
    && ls -l --block-size=M

# 结果
agentserver
sha256:5d91a7f659a141a4d00c504a3f0315026c70bde6e8ca7066d106df3e7f972e44
REPOSITORY          TAG              IMAGE ID       CREATED          SIZE
agent-server/cdh    6.3.2            5d91a7f659a1   8 seconds ago    2.8GB
master-server/cdh   6.3.2            d4f3e4ee3f9e   14 minutes ago   3.62GB
server/bashimage    latest           c8d140a652e0   29 minutes ago   404MB
centos              centos7.7.1908   08d05d1d5859   22 months ago    204MB
-rw------- 1 root root 2707M Sep 12 18:58 agent-server.tar.gz
...

有问题欢迎在评论区留言，你的提问就是对我最大的支持

secretWHD

关注

2
点赞
踩
5

收藏

觉得还不错? 一键收藏
打赏
0
评论
制作篇3 - 制作agent-server镜像包

1. 创建并运行agent容器docker run -d \--hostname agentserver \--name agentserver \--privileged=true \server/bashimage \/usr/sbin/init \&& docker ps结果：9bba31143dcf70a4775a7d20085ea277f940f67e132cfe9b8d7001fa92189d87CONTAINER ID IMAGE
复制链接

扫一扫