CM升级
- 运维:
-
- root统一密码
- 不要误删集群备份文件
- 登录cmserver安装的主机,执行命令:
- cat /etc/cloudera-scm-server/db.properties
- 登录postgresql数据库
-
- psql -U scm -p 7432 输入密码:
- 备份CM数据:
-
- pg_dump -h cdhmaster -p 7432 -U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)
- 检查/tmp下是否有文件生成,期间保证tmp下文件不要被删除。
- 停止impala hue hive 服务
- 停止CM server :
-
- sudo service cloudera-scm-server stop
- 停止CM server依赖的数据库:
-
- sudo service cloudera-scm-server-db stop
- 如果这台CM server上有agent在运行也停止:
-
- sudo service cloudera-scm-agent stop
- 修改yum的 cloudera-manager.repo文件:
-
- sudo vim /etc/yum.repos.d/cloudera-manager.repo
-
- [cloudera-manager]
# Packages for Cloudera Manager, Version 5, on RedHat or CentOS 6 x86_64
name=Cloudera Manager
baseurl=http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5/
gpgkey = http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck = 1
- [cloudera-manager]
- 安装:
-
- sudo yum clean all
sudo yum upgrade 'cloudera-*'
- sudo yum clean all
- 检查:
-
- rpm -qa 'cloudera-manager-*'
- 启动CM server 数据库:
-
- sudo service cloudera-scm-server-db start
- 启动CM server:
-
- sudo service cloudera-scm-server start
- 登录http://172.20.0.83:7180/
-
- 安装agent
- 升级如果升级jdk,会使hbase shell不可用,需要重新配置java_home
- 升级CM后需要重启CDH。
CDH升级
- 停止集群所有服务
- 备份namenode元数据:
-
- 进入namenode dir,执行:
-
- tar -cvf /root/nn_backup_data.tar ./*
- 下载parcels
- 分发包->激活包->关闭(非重启)
- 开启zk服务
- 进入HDFS服务->升级hdfs metadata
-
- namenode上启动元数据
- 启动剩余HDFS角色
- namenode响应RPC
- HDFS退出安全模式
- 备份hive metastore数据库
-
- mysqldump -h hostname -ucdhhive -p111111 cdhhive > /tmp/database-backup.sql
- 进入hive服务->更新hive metastore database scheme
- 更新oozie sharelib:oozie->install oozie share lib
-
- 创建 oozie user sharelib
- 创建 oozie user Dir
- 更新sqoop:进入sqoop服务->update sqoop
-
- 更新sqoop2 server
- 更新spark(略,可先卸载原来版本,升级后直接安装新版本)
- 启动集群所有服务:zk->hdfs->spark->flume->hbase->hive->impala->oozie->sqoop2->hue
- 分发客户端文件:deploy client configuration
-
- deploy hdfs client configuration
- deploy spark client configuration
- deploy hbase client configuration
- deploy yarn client configuration
- deploy hive client configuration
- 删除老版本包:
-
- sudo vim /etc/profile
- source /etc/profile
- jdk1.7.0_67-cloudera
-
- sudo yum remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
- 启动agent:
-
- sudo service cloudera-scm-agent restart
- HDFS metadata update
-
- hdfs server->instance->namenode=>action->Finalize Metadata Upgrade
准备工作:
- agent自动重启关闭:
- 事先下载好parcals包
- root统一密码
- 整理好需要备份哪些数据库以及备份命令