CDH6.1.1离线更新日志

CDH6.1.1离线更新日志

安装条件：服务器完全纳管，无root用户密码；无外网；数据库无root权限。
服务器：
SERVER01(cm server|mysql)
SERVER02(cm agent)
SERVER03(cm agent)
OS: RHEL 7.3

更新内容：CM/CDH
当前CM&CDH版本：6.0.1
目标CM&CDH版本：6.1.1

下载安装文件
CM安装文件：
https://archive.cloudera.com/cm6/6.1.1/redhat7/yum/
下载目录下文件
CDH安装文件：
https://archive.cloudera.com/cdh6/6.1.1/parcels/

检查当前版本信息
检查系统版本
lsb_release -a
redhat 7.3

数据库配置
cat /etc/cloudera-scm-server/db.properties
mysql

CM&CDH信息
http://server:7180/cmf/home
support>about

Version: Cloudera Express 6.0.1 
Java VM Name: Java HotSpot(TM) 64-Bit Server VM
Java VM Vendor: Oracle Corporation
Java Version: 1.8.0_141

备份CM

数据库信息
cat /etc/cloudera-scm-server/db.properties
或clusters>cloudera managerment service>configuration

instances信息
clusters>cloudera managerment service>instances

1. 备份agent

mkdir -p /opt/cm_backup/
cd /opt/cm_backup/
export CM_BACK_DIR="`date +%F`-CM6.0.1"
echo $CM_BACK_DIR
mkdir -p $CM_BACK_DIR

sudo -E tar -cf $CM_BACK_DIR/cloudera-scm-agent.tar --exclude=*.sock /etc/cloudera-scm-agent /etc/default/cloudera-scm-agent /var/run/cloudera-scm-agent /var/lib/cloudera-scm-agent

sudo -E tar -cf $CM_BACK_DIR/repository.tar /etc/yum.repos.d

2. 备份CM文件

sudo cp -rp /var/lib/cloudera-service-monitor /var/lib/cloudera-service-monitor-`date +%F`-CM6.0.1
sudo cp -rp /var/lib/cloudera-host-monitor /var/lib/cloudera-host-monitor-`date +%F`-CM6.0.1
sudo cp -rp /var/lib/cloudera-scm-eventserver /var/lib/cloudera-scm-eventserver-`date +%F`-CM6.0.1

3. Stop service

clusters>cloudera management service ---->stop
sudo systemctl stop cloudera-scm-server

4. 备份数据库

mysqldump --databases scm --host=localhost --port=3306 -u my_scm -p>scm-backup-`date +%F`-CM6.0.1.sql
mysqldump --databases hue --host=localhost --port=3306 -u my_hue -p>hue-backup-`date +%F`-CM6.0.1.sql
mysqldump --databases metastore --host=localhost --port=3306 -u my_metastore -p>metastore-backup-`date +%F`-CM6.0.1.sql
mysqldump --databases oozie --host=localhost --port=3306 -u my_oozie -p>oozie-backup-`date +%F`-CM6.0.1.sql

5. 备份CM文件(2)

sudo -E tar -cf $CM_BACK_DIR/cloudera-scm-server.tar --exclude=*.sock /etc/cloudera-scm-server /etc/default/cloudera-scm-server /var/run/cloudera-scm-server /var/lib/cloudera-scm-server

6. Start

sudo systemctl restart cloudera-scm-server
clusters>cloudera management service ---->start

升级CM

1. 配置yum
修改cloudera-manager.repo

[cloudera-manager]
name=Cloudera Manager 6.1.1
baseurl=http://server:8900/cloudera-repos/redhat7/yum/cm6.1.1
gpgkey=http://server:8900/cloudera-repos/redhat7/yum/cm6.1.1/RPM-GPG-KEY-cloudera
gpgcheck=1
enabled=1
autorefresh=0
type=rpm-md

拷贝导入repo

cp /var/www/html/cloudera-repos/redhat7/yum/cm6.1.1/cloudera-manager.repo /etc/yum.repos.d/
rpm --import /var/www/html/cloudera-repos/redhat7/yum/cm6.1.1/RPM-GPG-KEY-cloudera

2. 开启临时http服务

cd /var/www/html
python -m SimpleHTTPServer 8900

3. 检查依赖关系

yum deplist cloudera-manager-agent

4. JDK

这里更新update141到update181非必要，请自行选择

5. stop service

clusters>cloudera management service ---->stop
sudo systemctl stop cloudera-scm-server
sudo systemctl stop cloudera-scm-agent

6. yum install
[01]

sudo yum upgrade cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent

[02-03]

sudo yum upgrade cloudera-manager-daemons cloudera-manager-agent

7. service start

sudo systemctl start cloudera-scm-server
sudo systemctl start cloudera-scm-agent
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

8. 主机检查

重启成功后访问web
http://server:7180/cmf/upgrade
Run Host Inspector

Start Cloudera Management Service




home>cluster 1->restart
home>cluster 1->deploy client configuration

升级CDH

查看当前运行情况
hdfs fsck / -includeSnapshots
hdfs dfsadmin -report
hbase hbck

1. 备份cdh
zookeeper[01]

sudo cp -rp /var/lib/zookeeper /var/lib/zookeeper-`date +%F`-CM6.0.1

jn高可用：

sudo cp -rp /dfs/jn /dfs/jn-`date +%F`-CM6.0.1

nn[01]

mkdir -p /etc/hadoop/conf.rollback.namenode
cd /var/run/cloudera-scm-agent/process/ && cd `ls -t1 | grep -e "-NAMENODE\$" | head -1`
cp -rp * /etc/hadoop/conf.rollback.namenode/
rm -rf /etc/hadoop/conf.rollback.namenode/log4j.properties
cp -rp /etc/hadoop/conf.cloudera.hdfs/log4j.properties /etc/hadoop/conf.rollback.namenode/

dn[02-03]

mkdir -p /etc/hadoop/conf.rollback.datanode
cd /var/run/cloudera-scm-agent/process/ && cd `ls -t1 | grep -e "-DATANODE\$" | head -1`
cp -rp * /etc/hadoop/conf.rollback.datanode/
rm -rf /etc/hadoop/conf.rollback.datanode/log4j.properties
cp -rp /etc/hadoop/conf.cloudera.hdfs/log4j.properties /etc/hadoop/conf.rollback.datanode/

snn[01]非高可用

mkdir -p /etc/hadoop/conf.rollback.secondarynamenode
cd /var/run/cloudera-scm-agent/process/ && cd `ls -t1 | grep -e "-SECONDARYNAMENODE\$" | head -1`
cp -rp * /etc/hadoop/conf.rollback.secondarynamenode/
rm -rf /etc/hadoop/conf.rollback.secondarynamenode/log4j.properties
cp -rp /etc/hadoop/conf.cloudera.hdfs/log4j.properties /etc/hadoop/conf.rollback.secondarynamenode/

hue[01]

mkdir -p /opt/cloudera/parcels_backup
cp -rp /opt/cloudera/parcels/CDH/lib/hue/app.reg /opt/cloudera/parcels_backup/app.reg-`date +%F`-CM6.0.1

2. 进入维护模式
home>status->Enter Maintenance Mode

3. 拷贝parcel文件
查看parcels信息
hosts>parcels
记住local地址
Local Parcel Repository Path:/opt/cloudera/parcel-repo

parcel文件拷贝到/opt/cloudera/parcel-repo/下

mv CDH-6.1.1-1.cdh6.1.1.p0.875250-el7.parcel /opt/cloudera/parcel-repo/
mv CDH-6.1.1-1.cdh6.1.1.p0.875250-el7.parcel.sha /opt/cloudera/parcel-repo/
mv manifest.json /opt/cloudera/parcel-repo/

p.s.这里强调一下，原文件sha256需从manifest.json中选择对应合适的hash覆盖，且重命名为xxx.sha

授权并修改属主

chmod -R 775 *
chown -R cloudera-scm:cloudera-scm CDH-6.1.1-1.cdh6.1.1.p0.875250-el7.parcel CDH-6.1.1-1.cdh6.1.1.p0.875250-el7.parcel.sha manifest.json

重启cm

sudo systemctl restart cloudera-scm-server
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
sudo systemctl restart cloudera-scm-agent
tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

4. 在parcels页面点6.1.1的distribute，分发到各agent上

5. upgrade




6. 退出维护模式
home>status->Exit Maintenance Mode