Ambari--从零开始集成自定义组件

 

1. rpm包制作

作用：集成所有docker依赖离线安装包，安装脚本。

 

1.1 安装工具

安装rpm-build

yum install rpm-build -y

 

安装rpmdevtools

yum install rpmdevtools -y

 

1.2 创建工作空间

创建工作空间

rpmdev-setuptree -y

查看工作空间

 

创建文件夹

mkdir -p /root/rpmbuild/SOURCES/zoo/{bin,conf,lib}

1.3 拷贝所需文件

将docker安装包和安装脚本拷贝至lib文件夹下

Install.sh为安装脚本，内容如下

#!/bin/bash function install_docker(){   #卸载cgroups yum remove libcgroup -y   #关闭防火墙 systemctl stop firewalld systemctl disable firewalld   #关闭selinux setenforce 0 sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config   #开启服务器转发 modprobe br_netfilter echo "net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf sysctl -p   #关闭swap swapoff -a sed -i '/swap/s/^/#/g' /etc/fstab   #开启cgroups sed -i '/GRUB_CMDLINE_LINUX/d' /etc/default/grub echo 'GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1" GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"' >> /etc/default/grub   #安装docker yum localinstall /usr/hdp/3.0.1.0-187/zoo/lib/docker/*.rpm -y systemctl restart docker   } install_docker

修改文件权限

chmod +x install.sh

进入/root/rpmbuild/SOURCES目录

打包zoo压缩包

tar -cvzf zoo.tar.gz zoo/

 

1.4 编辑spec文件打包

进入/root/rpmbuild/SPECS目录

编写zoo.spec文件，内容如下

其中%install为执行安装的步骤

 

Name:         zoo Version:        1.0 # Avoid some silly dependencies and allow # symlink in the init.d folder AutoReqProv:    no Release:        0 Summary:        A unified Data Analytics and AI platform for distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray. License:        Apache 2.0 Group:          Applications/Internet Source0:        zoo.tar.gz Prefix:         /usr/hdp/3.0.1.0-187 BuildArch:      noarch BuildRoot:      %{_tmppath}/%{name}-root Provides:       zoo Requires(post): /sbin/chkconfig, /sbin/service %description A unified Data Analytics and AI platform for distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray     %prep #打包时不丢弃debug信息 %define __strip /bin/true #打包之前不要对目标文件做strip %define __os_install_post %{nil} #禁止自动搜索依赖关系 #Autoreq:no #Autoprov:no   %build   %install rm -rf $RPM_BUILD_ROOT mkdir -p $RPM_BUILD_ROOT/usr/hdp/3.0.1.0-187/zoo mkdir -p $RPM_BUILD_ROOT/etc/init.d mkdir -p %{_localstatedir}/lib/zoo tar xfz $RPM_SOURCE_DIR/zoo.tar.gz cp -r $RPM_BUILD_DIR/zoo/* $RPM_BUILD_ROOT/usr/hdp/3.0.1.0-187/zoo #%pre   %pre -p /bin/sh   %preun   %clean   #%post %post -p /bin/sh   %files %defattr(-,root,root,-) /usr/hdp/3.0.1.0-187/zoo   %changelog * Tue Aug 19 2014 - daniel.beneyto at abiquo dot com - Initial release.

 

1.5 执行打包命令

rpmbuild -ba zoo.spec

查看rpm包

cd /root/rpmbuild/RPMS/noarch

 

 

 

1. rpm本地缓存及仓库构建

作用:  使ambari执行yum install时使用本地rpm包

2.1 安装工具

安装createrepo

yum install createrepo -y

2.2 创建本地yum缓存地址

mkdir -p /var/www/html/zoo-repo/

复制rpm包至/var/www/html/zoo-repo/文件夹下

创建yum缓存文件

cd /var/www/html/zoo-repo/

createrepo ./

 

2.3 配置repo文件

在/etc/yum.repos.d/文件夹下创建zoo.repo，内容如下

 

[zoo] name=zoo baseurl=file:///var//www/html/zoo-repo gpagcheck=0 enabled=1

 

2.4 清除当前yum缓存并重新构建

yum clean all

yum makecache

 

 

1. 镜像制作

节点：10.180.210.37

作用：Intel Analytics-Zoo环境及Jupyter环境

3.1 Dockerfile

改动  删除启动脚本start-notebook.sh

修改entrypoint和cmd  支持参数传入

Dockerfile如下，没有配置环境变量，完整Dockerfile见10.180.210.37:/home/zoo/dockerfile/Dockerfile

FROM centos:centos7.4.1708 WORKDIR / RUN  mkdir -p /opt/workspace/ ADD  anaconda3.tar.gz /root/ ADD  test.tar.gz /opt/workspace/ COPY hadoop /usr/hdp/3.0.1.0-187/hadoop COPY spark2 /usr/hdp/3.0.1.0-187/spark2 COPY java   /usr/jdk64/java COPY etc/hadoop /etc/hadoop COPY hadoop-hdfs /usr/hdp/3.0.1.0-187/hadoop-hdfs COPY hadoop-mapreduce /usr/hdp/3.0.1.0-187/hadoop-mapreduce COPY hadoop-yarn /usr/hdp/3.0.1.0-187/hadoop-yarn COPY usr /usr/hdp/3.0.1.0-187/usr ENTRYPOINT ["jupyter","notebook","--allow-root","--no-browser","--ip='*'","--notebook-dir='/opt/workspace'"] CMD ["--port=12345","--NotebookApp.token=qwer1234"]

3.2 镜像构建和保存

docker build  -t zoo-insight-v4:v1.0 .

docker save zoo-insight-v4:v1.0 > zoo-InsightV4.tar

 

1. Ambari 集成

4.1 创建组件配置文件夹

在insightV4 manager节点执行如下命令

cd /var/lib/ambari-server/resources/stacks/HDP/3.0/services

创建文件夹

mkdir ZOO

创建如下文件夹

cd ZOO

下列所有操作均在ZOO文件夹及子文件夹内进行

mkdir -p  configuration quicklinks package/scripts

4.2 主配置文件metainfo.xml

创建组件主配置文件metainfo.xml

内容如下

<?xml version="1.0"?>   <!-- AnalyticsZoo metainfo -->   <metainfo>     <schemaVersion>1.9.0</schemaVersion>     <services>         <service>               <!--  组件名称-->             <name>ZOO</name>             <!--  组件显示名称-->             <displayName>AnalyticsZoo</displayName>             <!--  组件显示简介-->             <comment>Analytics-Zoo是基于spark的大数据分析+AI的端到端的分布式深度学习框架</comment>             <!--  组件显示版本-->     <version>1.0.0</version>             <components>                 <component>                     <name>ZOO</name>                     <displayName>Analytics-Zoo</displayName> <!-- 组件节点-->                     <category>MASTER</category>                     <!-- 组件节点个数-->                     <cardinality>1</cardinality>                     <versionAdvertised>true</versionAdvertised>                     <commandScript>                         <!-- 组件操作脚本 -->                         <script>scripts/zoo.py</script>                         <!-- 组件操作脚本语言 -->                         <scriptType>PYTHON</scriptType>                         <!-- 组件操作脚本执行超时时间 -->                        <timeout>1200</timeout>                     </commandScript>                     <logs>                         <log>                             <logId>analytics-zoo</logId>                             <primary>true</primary>                         </log>                     </logs>                 </component>             </components>             <osSpecifics>                 <osSpecific>                     <osFamily>any</osFamily>                     <packages>                         <package>                             <name>zoo</name>                         </package>                     </packages>                 </osSpecific>             </osSpecifics>             <configuration-dependencies>                 <!-- 组件参数配置文件-->                 <config-type>analytics-zoo-env</config-type>             </configuration-dependencies>             <quickLinksConfigurations>                 <quickLinksConfiguration>                     <!-- 组件快速链接配置文件-->                     <fileName>quicklinks.json</fileName>                     <default>true</default>                 </quickLinksConfiguration>             </quickLinksConfigurations>         </service>     </services> </metainfo>

4.3 配置组件参数配置文件

在configuration文件夹中创建analytics_zoo_env.xml内容如下：

<?xml version="1.0"?>   <?xml-stylesheet type-"text/xsl" href="configuration. xsl"?> <!-- analytics-zoo-env --> <configuration supports_adding_forbidden="true">     <property>         <name>notebook_port</name>         <display-name>notebook_port</display-name>         <value>12345</value>         <description>notebook_port</description>         <on-ambari-upgrade add="true"/>     </property>     <property>         <name>notebook_token</name>         <display-name>notebook_token</display-name>         <description>notebook 密码</description>         <value-attributes>             <type>password</type>             <overridable>false</overridable>         </value-attributes>         <on-ambari-upgrade add="true"/>     </property> </configuration>

 

4.4 Zoo容器化核心脚本，安装启停状态监测功能实现

在package/scripts文件夹下创建以下四个文件

params.py  读取页面配置及内置参数的脚本

start.sh   控制Zoo容器及docker服务启动的脚本

stop.sh   控制Zoo容器及docker服务停止的脚本

zoo.py    执行安装启停状态监测的脚本

   params.py内容如下

# -*- coding: utf-8 -*- #!/usr/bin/env python """ Zoo Params configurations """   # from resource_management.libraries.functions.version import format_hdp_stack_version, compare_versions from resource_management import * from resource_management.core.logger import Logger import os, re, json, functools import resource_management.libraries.functions from resource_management.libraries.functions.default import default from resource_management.libraries.functions.format import format from resource_management.libraries.script.script import Script from os.path import join   Logger.initialize_logger() config = Script.get_config() # Logger.info("config:{0}".format(config)) Logger.info('Starting config params ...')   root_user = 'root'   analytics_zoo_env = config['configurations']['analytics_zoo_env'] Logger.info("analytics_zoo_env:{0}".format(analytics_zoo_env))   notebook_port = config['configurations']['analytics_zoo_env']['notebook_port'] Logger.info("notebook_port:{0}".format(notebook_port))   notebook_token = config['configurations']['analytics_zoo_env']['notebook_token'] Logger.info("notebook_token:{0}".format(notebook_token))   zoo_log_dir = '/var/log/zoo/' zoo_log_file = 'zoo.log' zoo_containerid_file = "zoo.containerid" docker_pid_file = "docker.pid"   config = Script.get_config()

Zoo.py内容如下

# -*- coding: utf-8 -*-   from resource_management import * from resource_management.core.logger import Logger from resource_management.libraries.functions.check_process_status import check_process_status from resource_management.core.resources.system import Execute, File from resource_management.libraries.functions.show_logs import show_logs import os import socket     class Zoo(Script):     def configure(self, env):         import params         Logger.info('Configure Starting Config Params ...')         env.set_params(params)         Logger.info('Configure Config Params.')       def install(self, env):         import params         env.set_params(params)         Logger.info('Starting Install Analytics-Zoo ...')         self.install_packages(env)         install_zoo_cmd = "/usr/hdp/3.0.1.0-187/zoo/lib/install.sh".format()         Execute(install_zoo_cmd,user=params.root_user)         Logger.info('Installed Analytics-Zoo!')       def start(self, env):         import params         env.set_params(params)         Logger.info('Starting Analytics-Zoo ...')         notebook_port = params.notebook_port         notebook_token = params.notebook_token         start_zoo_cmd = "/var/lib/ambari-server/resources/stacks/HDP/3.0/services/ZOO/package/scripts/start.sh " +notebook_port + " " + notebook_token         Execute(start_zoo_cmd,user=params.root_user)         Logger.info('Started Analytics-Zoo !')       def stop(self, env):         stop_zoo_cmd = "/var/lib/ambari-server/resources/stacks/HDP/3.0/services/ZOO/package/scripts/stop.sh"         os.system(stop_zoo_cmd)         Logger.info('Stopping Analytics-Zoo')         def status(self, env):         import params         env.set_params(params)         Logger.info('Checked Analytics-Zoo Status')         check_process_status(os.path.join(params.zoo_log_dir, params.docker_pid_file))         Logger.info('Checked Analytics-Zoo Status')   if __name__ == "__main__":   Zoo().execute()

start.sh内容如下

#!/bin/bash   systemctl restart docker if[[ `docker images -q|grep c77c311e82ed` -ne 0 ]];then docker load -i /tmp/zoo-InsightV4.tar else echo "image is exist" fi   zoo_log_dir="/var/log/zoo" zoo_containerid_file="/var/log/zoo/zoo.containerid" docker_pid_file="/var/log/zoo/docker.pid"   if[ ! -d "$zoo_log_dir" ];then mkdir -p $zoo_log_dir fi   docker run -itd --net=host zoo-insight-v4:v1.0 --port=$1 --NotebookToken=$2   docker ps -q > $zoo_containerid_file   pgrep docker -o > $docker_pid_file

stop.sh内容如下

#!/bin/bash   docker stop $(docker ps -a -q)   docker rm $(docker ps -a -q)   zoo_log_dir="/var/log/zoo"   zoo_containerid_file="/var/log/zoo/zoo.containerid"   docker_pid_file="/var/log/zoo/docker.pid"   sleep 3   docker ps -q > $zoo_containerid_file   systemctl stop docker   sleep 5   pgrep docker -o > $docker_pid_file

 

1. 1. 快速链接配置

在quicklinks下创建quicklinks.json文件内容如下

快速链接端口需要与安装时配置端口相同。

{   "name": "default",   "description": "Analytics-Zoo-webUI default quick links configuration",   "configuration": {     "protocol":     {       "type":"http"     },       "links": [       {         "name": "Analytics-Zoo-webUI",         "label": "Analytics-Zoo webUI",         "requires_user_name": "false",         "component_name": "ZOO",         "url":"%@://%@:%@/login",         "port":{           "http_default_port": "30111",           "regex": "^(\\d+)$",           "site": "analytics_zoo_env"         }       }     ]   } }

 

5．重启Ambari安装组件

5.1 清除缓存，重启Ambari

Server1、Server2、Server3：

rm -rf /var/lib/ambari-agent/cache/*

 

重启

Server1：

ambari-server restart

 

Server1、Server2、Server3：

ambari-agent restart

5.2 上传镜像，启动组件

使用工具将zoo-InsightV4.tar上传至manager节点的/tmp文件夹下

在页面执行安装操作即可

重新制作镜像后需要把start.sh中镜像id修改为上传镜像的id