基于openEuler22.03-sp3（ARM）部署hdp3.3.2

 

背景介绍

# 服务器示例

10.25.188.127 ambari
10.25.188.128 node1
10.25.188.129 node2

# 本次实验基于以下环境

# 操作系统：openEuler22.03-sp3（ARM）
# hdp版本：HDP3.3.2.0-008(aarch64)
# 参考网站:http://www.hdp.link/
# 版本及兼容说明：http://mp.weixin.qq.com/s?__biz=MzkwODM4NTc4Mw==&mid=2247484233&idx=1&sn=3ca3487679df6a7c211255073c692069&chksm=c0cb8fd1f7bc06c744cf3d9f618a58e359eeea58776f4f8304fbce755f0b1e2e786a9f6a99fa&mpshare=1&scene=1&srcid=05066nFjnB7CSPQAG8brqjZ2&sharer_shareinfo=759a73c3afa1be4d43f0879f134b8c35&sharer_shareinfo_first=759a73c3afa1be4d43f0879f134b8c35#rd

```

#一、基础环境部署

1.1 安装基础组件
```
yum -y install vim wget net-tools tree createrepo unzip openssl

mkdir -p /data/{pkg,iso,software,mysql}
```
1.2 配置hostname
```
hostnamectl set-hostname ambari
hostnamectl set-hostname node1
hostnamectl set-hostname node2
```
1.3 配置静态IP
```
# vim /etc/sysconfig/network-scripts/ifcfg-xxx

# ONBOOT="yes" # 设置开机启动，确保网络配置有效，否则无法建立网络连接
# IPADDR="192.168.5.10" # 静态IP地址
# PREFIX="24" # 子网掩码，相当于255.255.255.0
# GATEWAY="192.168.5.1" # 默认网关IP地址
# DNS1="192.168.1.1" # 首选DNS服务器IP地址，上网必备
```
1.4 优化ssh连接
```
#选做
```
1.5 配置hosts
```
cat >> /etc/hosts <<eof
10.25.188.127 ambari
10.25.188.128 node1
10.25.188.129 node2
eof

cat /etc/hosts
```
1.6 配置时间同步
```
yum -y install chrony
```
```
参考部署
#https://cloud.tencent.com/document/product/213/53629

#本次选的是以下时间服务器：

#111.230.189.174
#120.25.115.20
```
```
cat /etc/chrony.conf

cat >> /etc/chrony.conf<<eof
pool 111.230.189.174 iburst
pool 120.25.115.20 iburst
eof

systemctl enable --now  chronyd
systemctl is-active chronyd
#同步时间
chronyc -a makestep
```
```
#修改时区的操作：

cat >> /etc/profile <<eof
TZ='Asia/Shanghai'; export TZ
eof

cat /etc/profile
source /etc/profile

#参考http://www.manongjc.com/detail/57-uvmdjxzmjwmvyaz.html

#openeuler 官网时区设置

# timedatectl list-timezones
# timedatectl set-timezone Asia/Shanghai

```

1.7 关闭防火墙和selinux
```
systemctl status firewalld
getenforce

systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
setenforce 0

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

cat /etc/selinux/config
```

1.8 修改最大打开文件数量限制和修改umask值

```
#官网建议打开文件描述符的最大数量为 10000 或更多

#查看软硬链接
ulimit -Sn
ulimit -Hn

#查看 umask
umask
```
```
临时修改
ulimit -n 10000
```

```
#长期生效

ll /etc/security/limits.conf

cat >> /etc/security/limits.conf <<eof
#表示任何一个用户可以打开的最大的文件描述符数量 
* soft nofile 655350 
* hard nofile 655350 
eof
#hard limit不能大于/proc/sys/fs/nr_open，如果hard limit大于nr_open，注销后将无法正常登录。
```

```
#加载配置
source /etc/profile
```
1.9 安装jdk
```
#上传jdk安装包

#解压

#本次以：jdk-8u411-linux-aarch64.tar.gz 安装包来部署jdk

tar -zxvf jdk-8u411-linux-aarch64.tar.gz -C /usr/local/
ln -s /usr/local/jdk1.8.0_411/ /usr/local/jdk

#配置环境变量
cat >> /etc/profile << 'EOF'
#Set for java
export JAVA_HOME=/usr/local/jdk
export PATH=$PATH:$JAVA_HOME/bin
EOF

#加载环境变量
source /etc/profile
```
```
#Centos安装openjdk11并配置JAVA_HOME
#https://blog.csdn.net/qq_29752857/article/details/132655351
```

1.10 安装httpd

```
yum -y install httpd
#mkdir /var/www/html/{ambari,hdp}
systemctl enable --now httpd
systemctl status httpd

```

1.11 配置MySQL的Java调用工具
```
# 安装mysql-connector-java，安装包：mysql-connector-java-5.1.48.jar

mkdir -p /usr/share/java/
mv mysql-connector-java-5.1.48.jar /usr/share/java/mysql-connector-java.jar

```

#二、本地repo源部署

2.1 配置操作系统本地镜像yum源 （在线可忽略）

```
#挂载镜像

cat >> /etc/fstab << 'EOF'
#参考示例，以实际镜像为准
#/media/CentOS-7-x86_64-Everything-2009.iso /mnt iso9660 defaults 0 0
EOF

#mount -a

#本地镜像repo示例

cat >> /etc/yum.repos.d/local.repo << 'EOF'
[dvd]
name=dvd
baseurl=file:///data/iso
enabled=1
gpgcheck=0
EOF
```

2.2 准备安装包并上传到服务器并配置本地amabari和hdp的repo

`注意`文件比较大，建议提前上传
```
2.2.1 上传并解压安装包到指定路径，

#hdp相关的离线安装包存放在此 /data/pkg 目录

ln -s /data/pkg /var/www/html/

cd /data/pkg

#解压相关安装包

#amabari包
tar -xf ambari-2.7.6.0-28-redhat9-aarch64.tar.gz -C /data/pkg

#HDP
tar -xf HDP-3.3.2.0-008-redhat789-aarch64.tar.gz -C /data/pkg
 
#HDP-GPL
tar -xf HDP-GPL-3.3.2.0-008-redhat789-aarch64.tar.gz -C /data/pkg

#HDP-UTILS
tar -xf HDP-UTILS-1.1.0.22-redhat789-aarch64.tar.gz -C /data/pkg

#其他包
tar -xf psycopg2_centos7_aarch64.tar.gz -C /data/pkg

mv sh-utils-1.0.0-001.aarch64.rpm /data/pkg/psycopg2_centos7/

cd /data/pkg/psycopg2_centos7/ && createrepo .

chmod -R 777 /data/pkg
```
```
#hdp v3.3.2 对应文件路径

# #ambari
# http://10.25.188.127/pkg/2.7.6.0-28/

# #hdp
# http://10.25.188.127/pkg/3.3.2.0-008/

# #HDP-UTILS
# http://10.25.188.127/pkg/HDP-UTILS/centos7/1.1.0.22/

# #gpl-3.3.2.0-008
# http://10.25.188.127/pkg/gpl-3.3.2.0-008/

# #psycopg2_centos7
# http://10.25.188.127/pkg/psycopg2_centos7/

```
```
2.2.2 配置Ambari HDP本地yum源

#注意：需要修改以实际的安装包

#ambari
cat > /etc/yum.repos.d/ambari.repo << 'EOF'
[Ambari]
name=Ambari
baseurl=http://10.25.188.127/pkg/2.7.6.0-28/
gpgcheck=0
enabled=1
priority=1
EOF

#hdp repo

cat > /etc/yum.repos.d/hdp.repo << 'EOF'
#VERSION_NUMBER=3.3.2
[HDP]
name=HDP
baseurl=http://10.25.188.127/pkg/3.3.2.0-008/
gpgcheck=0
enabled=1
priority=1

[HDP-GPL]
name=HDP-GPL
baseurl=http://10.25.188.127/pkg/gpl-3.3.2.0-008/
gpgcheck=0
enabled=1
priority=1

[HDP-UTILS]
name=HDP-UTILS
baseurl=http://10.25.188.127/pkg/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=0
enabled=1
priority=1

[xxx]
name=xxx
baseurl=http://10.25.188.127/pkg/psycopg2_centos7/
gpgcheck=0
enabled=1
priority=1

EOF

yum clean all
yum makecache
```

# 三、MySQL部署

```
#使用docker部署mysql5.7
```

3.1 安装docker
```

#下载阿里云docker yum 源
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo

#将docker-ce.repo中[docker-ce-stable]中的$releasever改为8
#替换$releasever为8

sed -i s#\$releasever#8#g   /etc/yum.repos.d/docker-ce.repo

#安装依赖包
yum clean all && yum makecache

mkdir -p /data/{mysql,docker}
cd       /data/mysql

yum install -y  device-mapper-persistent-data lvm2
```
```
#安装最新版本

yum -y install docker-ce
```

```
#设置开机启动docker
systemctl start docker
systemctl enable --now docker

#设置docker的cgroup驱动
#这里推荐systemd
cat > /etc/docker/daemon.json <<EOF
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "data-root": "/data/docker",
  "storage-driver": "overlay2"
}
EOF

# 使配置生效，重启docker
systemctl daemon-reload
systemctl restart docker

#查看docker版本
docker version
```
```
#参考：

# AMR架构下openEuler系统安装Docker和Kubernetes教程
# https://blog.csdn.net/southwalls/article/details/135689353
```

3.2 docker部署ARM架构下的MySQL5.7
```
docker pull biarms/mysql:5.7.30-linux-arm64v8

#保存为离线镜像
#docker save biarms/mysql:5.7.30-linux-arm64v8 -o mysql57.tar

mkdir -p /data/mysql/{conf,logs,data}

docker run -p 3306:3306 --name mysql57 \
-e MYSQL_ROOT_PASSWORD=Mysql_666 \
-d biarms/mysql:5.7.30-linux-arm64v8

docker cp mysql57:/etc/mysql/mysql.conf.d/mysqld.cnf  /data/mysql/conf/mysqld.cnf

docker rm -f mysql57

# 修改配置文件/data/mysql/conf/mysqld.cnf，在[mysqld]下增加skip-ssl，如下

# [mysqld]
# skip-ssl

sed -i  '/[mysqld]/a\skip-ssl' mysqld.cnf

docker run -p 3306:3306 --name mysql57 \
-v /data/mysql/conf/mysqld.cnf:/etc/mysql/mysql.conf.d/mysqld.cnf \
-v /data/mysql/logs:/var/log/mysql \
-v /data/mysql/data:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=Mysql_666 \
--restart=always \
-d biarms/mysql:5.7.30-linux-arm64v8
```
```
# 参考
# ARM架构下的Mysql5.7
# https://blog.csdn.net/qsm44593342/article/details/132775772
```

初始化MySQL
```
# -- 配置ambari库

create database ambari default charset utf8;

use ambari;

create user 'ambari'@'%' identified by 'Mysql_666';

grant all privileges on *.* to 'ambari'@'%';

create user 'ambari'@'localhost' identified by 'Mysql_666';

grant all privileges on *.* to 'ambari'@'localhost';

create user 'ambari'@'ambari' identified by 'Mysql_666';

grant all privileges on *.* to 'ambari'@'%';

# -- 配置hive库

create database hive default charset utf8;

create user 'hive'@'%' identified by 'Mysql_666';

grant all privileges on *.* to 'hive'@'%';

create user 'hive'@'localhost' identified by 'Mysql_666';

grant all privileges on *.* to 'hive'@'localhost';

create user 'hive'@'ambari' identified by 'Mysql_666';

grant all privileges on *.* to 'hive'@'ambari';

# -- 配置ozzie库

create database ozzie default charset utf8;

create user 'ozzie'@'%' identified by 'Mysql_666';

grant all privileges on *.* to 'ozzie'@'%';

create user 'ozzie'@'localhost' identified by 'Mysql_666';

grant all privileges on *.* to 'ozzie'@'localhost';

create user 'ozzie'@'ambari' identified by 'Mysql_666';

grant all privileges on *.* to 'ozzie'@'ambari';

-- 重新加载权限

GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'192.168.0.21' IDENTIFIED BY 'Mysql_666';
FLUSH PRIVILEGES;

# 查看创建的用户

select user,host, from mysql.user;

```

#四、部署并初始化 Ambari-server服务

4.1 安装Ambari-server服务
```
yum -y install ambari-server

#验证安装成功
rpm -qa | grep ambari
```

4.2 修改Ambari 配置，初始化MySQL数据库：
```
#ll /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

#进入MySQL，执行文件/var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

```

4.3 初始化ambari-server
```
[root@ambari ~]# ambari-server setup
...
Customize user account for ambari-server daemon [y/n] (n)? n
...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
...
Path to JAVA_HOME: /usr/local/jdk1.8.0_371
...
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
...
Enter advanced database configuration [y/n] (n)? y
...
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): ambari   
Port (3306): 3306
Database name (ambari): ambari
Username (ambari): ambari
Enter Database Password (bigdata): 
Re-enter password: 
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.5.0.72.jar
....
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

```

4.4 启动 Ambari-server

```
ambari-server start
```
```
#查看日志
tail -20f /var/log/ambari-server/ambari-server.log

如果部署成功：则会启动8080的端口，

ss -tnl
```

4.5 浏览器访问

```
http://192.168.81.200:8080/

账号密码: admin / admin
```

#五、部署报错及解决办法

5.1、hive连接数据库报错，提示jdbc未设置，
```
执行以下命令解决：

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
```

5.2、报错及解决办法：

```
#Problem: 无法为该任务安装最佳候选

#  - 没有东西可提供 /usr/bin/lsb_release（hadoop_3_3_2_0_008-3.3.6.3.3.2.0-008.aarch64 需要）

#原因：lsb没有安装，需要安装lsb，openEuler系统对应的是：openeuler-lsb

rpm -qa | grep lsb

#需要安装lsb，openEuler系统对应的是：openeuler-lsb-5.0-1.oe2203sp3.aarch64

yum -y install openeuler-lsb #需要repo支持，最好可联网

```

5.3、服务器初始化安装超时：
```

# 在服务器没有网络的情况下，需要删除在线的repo，否则，会优先调用在线的repo源
```

5.4、初始化的时候，ssh无法连接到node节点
```

需要将id_rs保存到本地，直接使用文件来输入id_rsa，直接粘贴的有点问题

```

#六、参考文章

6.1 【麒麟系统安装HDP，报错处理】更新篇
```
https://blog.csdn.net/weixin_44385419/article/details/132070562
```
6.2 进行HDP安装部署时在confirm hosts时卡在preparing的解决办法
```
https://blog.csdn.net/qq_41805514/article/details/88886015
```
6.3 Ambari安装hive服务，数据库Connection Failed

```
https://cloud.tencent.com/developer/article/2165983
```
#七、总结

1、ARM部署hdp，因为生态不成熟，部署起来，还是存在一定难度，但是逐步琢磨，问题还是可以解决的！

2、感谢[HiDataPlus]团队提供的hdp产品，同时在部署HDP中提供的支持与帮助!

3、大家部署有什么问题，可以来找我沟通交流！