使用Ansible在Centos7.9/龙晰Anolis7.9系统上离线安装CDH6.3.2

maoerniu01

已于 2022-11-24 16:40:33 修改

阅读量2.2k

点赞数 1

文章标签：大数据

于 2022-07-13 11:21:03 首次发布

本文链接：https://blog.csdn.net/weixin_40880094/article/details/125742894

版权

一、部署简介

总共3台机器，也可以将数据库单独部署

还有一台ansible主机，我这里用的是自己的笔记本， ansible安装略

Inventory

[cloudera_manager]
172.16.33.110 hostname=nn1

[cloudera_agent]
172.16.33.110 hostname=cdh01
172.16.33.111 hostname=cdh02
172.16.1.19 hostname=cdh03

[db_mysql]
172.16.33.110 hostname=cdh01

[all:children]
cloudera_manager
cloudera_agent
db_mysql

[all:vars]
ansible_connection=ssh
ansible_ssh_user=root
ansible_ssh_pass='123456'
ansible_sudo_pass='123456'

大致步骤如下：

1、系统初始化及一些基本配置

2、安装数据库，并对数据库进行配置，创建cloudera manager需要的一些数据库

3、架设离线安装源

4、安装cloudera manager

5、安装cloudera agent

6、打开CDH管理页面，创建集群

二、角色介绍

ansible 的几个role 介绍

角色的目录结构可以由 ansible-galaxy 自动生成，如：

ansible-galaxy init common

common

角色 common 的目录结构如下

erniudeMacBook-Pro:csdn erniu$ tree common
common
├── README.md
├── defaults
│   └── main.yml
├── files
│   ├── history.sh
│   └── jdk-8u331-linux-x64.rpm
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

其中主要任务文件 tasks/main.yml 如下

---
- name: Disable firewalld
  service:
    name: firewalld
    state: stopped
    enabled: no
- name: disable selinux
  lineinfile:
    path: /etc/selinux/config
    regexp: '^SELINUX='
    line: SELINUX=disabled
- name: Set time info in history command
  copy:
    src: history.sh
    dest: /etc/profile.d/
    mode: '0644'
- name: Set hostname
  hostname: name={{ hostname }}
- name: Install epel repo
  yum:
    name: epel-release
    state: present
- name: Update by yum
  yum:
    name: '*'
    state: latest
- name: Install common utils
  yum:
    name: "{{ packages }}"
    state: present
  vars:
    packages:
      - wget
      - net-tools
      - ack
      - telnet
      - vim
      - ntp
- name: Upload JDK
  copy:
    src: jdk-8u331-linux-x64.rpm
    dest: /usr/local/src/
- name: Install JDK from rpm
  yum:
    name: /usr/local/src/jdk-8u331-linux-x64.rpm
    state: present

files/history.sh

export HISTTIMEFORMAT="[%F %T] "

files/jdk-8u331-linux-x64.rpm 文件可以在oracle 官网上下载

还需要一个 role 的控制文件，和roles目录平级

common.yml

---
- name: Init OS
  hosts: '{{ hosts }}'
  gather_facts: true
  become: true
  become_method: sudo
  roles:
    - common

mysql

角色 mysql 的目录结构：

erniudeMacBook-Pro:roles erniu$ tree mysql
mysql
├── README.md
├── defaults
│   └── main.yml
├── files
│   └── my.cnf
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── tasks
│   ├── configure.yml
│   ├── install.yml
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

tasks/main.yml

---
- include_tasks: install.yml
- include_tasks: configure.yml

tasks/install.yml

---
- name: Remove mariadb-libs
  yum:
    name: mariadb-libs
    state: absent
- name: Install mysql rpm from remote repo
  yum:
    name: http://repo.mysql.com/mysql57-community-release-el7-10.noarch.rpm
    state: present
- name: Undo gpgcheck
  shell: sed -i 's/gpgcheck=1/gpgcheck=0/g' /etc/yum.repos.d/mysql-community.repo
- name: Install mysql
  yum:
    name: mysql-server
    state: present
- name: Start mysqld
  systemd:
    name: mysqld
    state: started
    enabled: yes

tasks/configure.yml

---
- name: Stop mysqld
  systemd:
    name: mysqld
    state: stopped
- name: Create a backup dir
  file: 
    path: /opt/backup/mysql
    state: directory
- name: Copy old InnoDB logfile0
  copy:
    src: /var/lib/mysql/ib_logfile0
    dest: /opt/backup/mysql/
    remote_src: yes
    force: no
- name: Copy old InnoDB logfile1
  copy:
    src: /var/lib/mysql/ib_logfile1
    dest: /opt/backup/mysql/
    remote_src: yes
    force: no
- name: Remove old InnoDB logfile0
  file:
    path: /var/lib/mysql/ib_logfile0
    state: absent
- name: Remove old InnoDB logfile1
  file:
    path: /var/lib/mysql/ib_logfile1
    state: absent
- name: Upload my.cnf
  copy:
    src: my.cnf
    dest: /etc/my.cnf
    mode: 0644
    owner: root
    group: root
  notify: restart mysql

handlers/main.yml

---
# handlers file for test_roles
- name: restart mysql
  systemd: 
    name: mysqld
    state: restarted

files/my.cnf

其中官方强调，explicit_defaults_for_timestamp=0 一定要设置为0 ，否则会报错

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
character-set-server = utf8
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0

key_buffer_size = 32M
max_allowed_packet = 16M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 600
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES


# cloudera
explicit_defaults_for_timestamp=0

同样，我们还需要一个调用 mysql 角色的 mysql.yml 文件，和 roles 目录平级

mysql.yml

---
- name: Install mysql
  hosts: '{{ hosts }}'
  user: root
  become: yes
  gather_facts: yes
  roles:
    - mysql

clouderamanager

clouderamanager 角色的目录结构

erniudeMacBook-Pro:roles erniu$ tree clouderamanager/clouderamanager/
├── README.md
├── defaults
│   └── main.yml
├── files
│   ├── cdh6
│   │   ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
│   │   ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
│   │   ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
│   │   └── manifest.json
│   ├── cm.repo
│   ├── cm6
│   │   ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
│   │   ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
│   │   ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
│   │   ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
│   │   ├── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
│   │   └── oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
│   └── mysql-connector-java.jar
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

这里files目录下的cdh6和cm6源文件，官方已经不支持下载了，需要的朋友可以加我微信，centos和ubuntu的都有

tasks/main.yml

---
- name: Add hosts dns
  blockinfile:
    path: /etc/hosts
    block: |
      {{ item.ip }}  {{ item.name }}
    marker: "# {mark} ANSIBLE MANAGED BLOCK {{ item.name }}"
  loop:
    - { name: cdh01, ip: 172.16.33.101 }
    - { name: chd02, ip: 172.16.33.102 }
    - { name: cdh03, ip: 172.16.1.18 }
- name: Install Apache
  apt: name=apache2 update_cache=yes state=latest
- name: Start Apache
  systemd:
    name: apache2
    state: started
    enabled: yes
- name: Upload server ntp.conf
  copy:
    src: ntp.conf
    dest: /etc/
    mode: '0644'
- name: Start ntp service
  service:
    name: ntp
    state: restarted
    enabled: yes
- name: Upload cdh files
  copy:
    src: cdh6
    dest: /var/www/html/
- name: Upload cloudear manager files
  copy:
    src: cm6
    dest: /var/www/html/
- name: Install createrepo
  apt:
    name: createrepo
    state: present
- name: Create cm repo
  command:
    cmd: createrepo .
    chdir: /var/www/html/cm6
- name: Add cm yum repo
  copy:
    src: cm.list
    dest: /etc/apt/sources.list.d/
    mode: '0644'
- name: Add archive.key
  apt_key:
    url: http://cdh01/cm6/6.3.1/ubuntu1804/apt/archive.key
    state: present
- name: Apt update
  apt:
    update_cache: true
- name: Install oracle-j2sdk1.8
  apt:
    name: oracle-j2sdk1.8
    state: present
- name: Install cloudera-manager server
  apt:
    name:
      - cloudera-manager-server
      # - cloudera-manager-server-db-2
    state: present
- name: Install libmysql-java
  apt:
    name: libmysql-java
    state: present
- name: Start cloudera manager
  service:
    name: cloudera-scm-server
    state: restarted
    enabled: yes

上面的hosts配置需要改成自己的

files/cm.repo

[CM]
name=cm6
baseurl=http://cdh01/cm6/
gpgcheck=0

调用clouderamanager角色的 clouderamanager.yml

---
- name: Install clouderamanager
  hosts: '{{ hosts }}'
  user: root
  become: yes
  gather_facts: yes
  roles:
    - clouderamanager

clouderaagent

clouderagent 角色的目录结构

erniudeMacBook-Pro:roles erniu$ tree clouderaagent/
clouderaagent/
├── README.md
├── defaults
│   └── main.yml
├── files
│   ├── cm.repo
│   └── mysql-connector-java.jar
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

tasks/main.yml

---
- name: Add hosts dns
  blockinfile:
    path: /etc/hosts
    block: |
      {{ item.ip }}  {{ item.name }}
    marker: "# {mark} ANSIBLE MANAGED BLOCK {{ item.name }}"
  loop:
    - { name: cdh01, ip: 172.16.33.110 }
    - { name: chd02, ip: 172.16.33.111 }
    - { name: cdh03, ip: 172.16.1.19 }
- name: Add cm yum repo
  copy:
    src: cm.repo
    dest: /etc/yum.repos.d/
    mode: '0644'
- name: Install packages
  yum:
    name: "{{packages}}"
  vars:
    packages:
    - libxslt
    - cyrus-sasl-plain
    - cyrus-sasl-gssapi
    - portmap
    - fuse-libs
    - /lib/lsb/init-functions
    - mod_ssl
    - openssl-devel
    - python3-psycopg2
    - fuse
- name: Upload mysql jdbc
  copy:
    src: mysql-connector-java.jar
    dest: /usr/share/java/
    mode: '0644'
- name: Install cloudera-manager agent
  yum:
    name:
      - cloudera-manager-daemons
      - cloudera-manager-agent
    state: present
- name: Change cloudera agent config
  lineinfile:
    path: /etc/cloudera-scm-agent/config.ini
    regexp: '^server_host='
    line: server_host=cdh01
- name: Start cloudera agent
  service:
    name: cloudera-scm-agent
    state: restarted
    enabled: yes

调用clouderaagent角色的 clouderaagent.yml

---
- name: Install clouderaagent
  hosts: '{{ hosts }}'
  user: root
  become: yes
  gather_facts: yes
  roles:
    - clouderaagent

三、部署服务

部署 common

common是基础设置，所以对所有机器都要执行

ansible-playbook -i host common.yml -e "hosts=all"

部署 mysql

仅对mysql服务器部署

ansible-playbook -i host mysql.yml -e "hosts=db_mysql"

配置mysql

查看密码：

grep 'temporary password' /var/log/mysqld.log

登录数据库，设置 root 密码：

mysql -uroot -p
ALTER USER 'root'@'localhost' IDENTIFIED BY 'MyNewPass4!';

降低密码复杂度：（可选）

SET GLOBAL validate_password.length = 6; 
SET GLOBAL validate_password.number_count = 0;

创建数据库和用户，按需选择

# Cloudera Manager Server
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456';

# Activity Monitor
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456';

# Reports Manager
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456';

# Hue
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456';

# Hive Metastore Server
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY '123456';

# Sentry Server
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456';

# Oozie
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456';

部署clouderamanager

ansible-playbook -i host clouderamanager.yml -e "hosts=cloudera_manager"

查看 clouderamanager 状态

systemctl status cloudera-scm-server

初始化数据库：

参数参考：

/opt/cloudera/cm/schema/scm_prepare_database.sh [options] <databaseType> <databaseName> <databaseUser> <password>

Step 5: Set up the Cloudera Manager Database | 6.3.x | Cloudera Documentation

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h localhost -uroot -p123456 --scm-host localhost scm scm 123456

补：如果数据库和cloudera-manager不在一个机器上，则

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h 192.168.1.20 --scm-host cdhm scm scm 123456

这里192.168.1.20是数据库的地址， cdhm是cloudera-manager本机的hostname

部署clouderaagent

ansible-playbook -i host clouderaagent.yml -e "hosts=cloudera_agent"

查看 clouderaagent状态

systemctl status cloudera-agent-scm-agent

在cdh01上

创建parcel-repo并重启cloudera-scm-server

Centos7.9

cp /var/www/html/cdh6/* /opt/cloudera/parcel-repo/

cd /opt/cloudera/parcel-repo/

mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

Anolis7.9

由于下载的CDH版本是Centos7的，而操作系统是龙晰操作系统，并不是Centos7。CM在分配CDH安装包是会校验操作系统版本，主要是指安装包后缀 el7 / el6等，龙晰操作系统后缀是an，不在CDH支持列表中。这种方式不是直接获取操作系统信息，故而不能通过修改/etc/redhat-release解决

cp /var/www/html/cdh6/* /opt/cloudera/parcel-repo/

cd /opt/cloudera/parcel-repo/

cp CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel CDH-6.3.2-1.cdh6.3.2.p0.1605554-unknown.parcel

mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

cp CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha CDH-6.3.2-1.cdh6.3.2.p0.1605554-unknown.parcel.sha

四、创建集群

浏览器登录 http://172.16.33.101:7180/

默认用户名密码 admin admin

按提示操作，这里选择免费版

点击继续后，过一会儿会跳到集群创建的步骤

设置集群名称

这里已经识别到了3台主机，全部勾选，并点击继续

已经配置了parcel源

安装 Parcels

选择接受

选择自定义服务，勾选你要安装的服务

下面是我的选择，也可以不选，后面再按需一个个添加

自定义角色分配，按自己的分配

数据库设置

这里的主机名称是你数据库所在的主机名，数据库名称和用户名密码，是刚才创建的

审核更改

如果需要改配置，可以在这一步更改

部署完成

这样集群就创建好了

wechat:

maoerniu01

关注

1
点赞
踩
3

收藏

觉得还不错? 一键收藏
2
评论
使用Ansible在Centos7.9/龙晰Anolis7.9系统上离线安装CDH6.3.2

使用Ansible在Centos7.9/龙晰Anolis7.9系统上离线安装CDH6.3.2
复制链接

扫一扫