mysql5.7.36环境建设

 主要记录Mysql系列的二进制部署、主从复制、xtrabackup、ansible-playbook、Prometheus入门操作,让新攻城狮成员遇到Mysql不慌,加油攻城狮!

教学视频:

2022最新-mysql5.7.36二进制包部署

2022最新-mysql5.7.36二进制包部署_哔哩哔哩_bilibili
2022最新-02mysql5.7.36xtrabackup全备还原_哔哩哔哩_bilibili
2022最新-03mysql5.7.36主从配置_哔哩哔哩_bilibili
2022最新-04mysql5.7.36xtrabackup增量还原_哔哩哔哩_bilibili

目录

一、准备

1.1、包下载地址

1.2、依赖检查

1.3、时间同步

二、Mysql环境安装

2.1、创建系统用户

2.2、创建目录并授权

2.3、安装包下载及解压

2.4、修改配置文件

2.5、配置环境变量

2.6、初始化数据库

2.7、配置system管理文件

2.8、服务启停

三、xtrabackup操作数据库全量备份及还原

3.1、数据库备份

3.2、处理事务文件

3.3、修改my.cnf数据文件路径

3.4、授权

3.5、重启数据库

四、Mysql主从配置

4.1、创建主从同步用户并授权

4.2、主从同步

4.3、启动并检查主从状态

五、xtrabackup操作数据库增量备份及还原

5.1、操作数据全量备份

5.2、按全量数据进行增量备份

5.3、处理全量数据的事务

5.4、合并数据

六、ansible-playbook自动化部署Mysql

6.1、ansible安装

6.2、配置ansible-playbook

七、prometheus监控Mysql

7.1、包下载

7.2、Systemd服务管理



        一、准备

系统:Centos 7.6

Mysql:5.7.36

时区:Asia/Shanghai

Selinux:关闭

Firewalld:关闭

Openfiles:65535

DefaultLimit:65535

1.1、包下载地址

Mysql Package

https://cdn.mysql.com/archives/mysql-5.7/mysql-5.7.36-el7-x86_64.tar.gz

Xtarbackup

https://www.percona.com/downloads/Percona-XtraBackup-2.4/LATEST/

Node_exporter

https://github.com/prometheus/mysqld_exporter/releases/download/v0.14.0/mysqld_exporter-0.14.0.linux-amd64.tar.gz

1.2、依赖检查

检查命令:rpm -qa | grep mariadb

yum remove mariadb

1.3、时间同步

安装

yum -y install ntp

刷新同步

ntpq -p

二、Mysql环境安装

2.1、创建系统用户

useradd --system mysql

2.2、创建目录并授权

mkdir -p /data/mysql/{binlogs,dbfiles,logs,relays}
chown -R mysql. /data/mysql

2.3、安装包下载及解压

wget https://cdn.mysql.com/archives/mysql-5.7/mysql-5.7.36-el7-x86_64.tar.gz && tar xf mysql-5.7.36-el7-x86_64.tar.gz -C /usr/local/

2.4、修改配置文件

#MySQL_version = 5.7.36
[client]
socket=/data/mysql/mysql.sock
​
[mysqld]
port = 3306
user = mysql
basedir = /usr/local/mysql-5.7.36
datadir = /data/mysql/dbfiles
tmpdir=/data/mysql
socket = /data/mysql/mysql.sock
pid-file=/data/mysql/mysql.pid
secure-file-priv=''
default_authentication_plugin = mysql_native_password
​
lower_case_table_names=1
​
server-id = 1
transaction_isolation = READ-COMMITTED
autocommit = 1
character_set_server=utf8mb4
max_connections = 4000
max_connect_errors = 1844674407370954751
connect_timeout = 10
lock_wait_timeout=3600
thread_cache_size=256
sql_mode = "NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER"
​
​
join_buffer_size = 4M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
max_allowed_packet = 16M
sort_buffer_size = 16M
​
########basic settings########
skip_name_resolve = 1
explicit_defaults_for_timestamp = 1
tmp_table_size = 67108864
​
​
########log settings########
log_error = /data/mysql/logs/mysql_err.log
log_timestamps = system
slow_query_log = 1
slow_query_log_file = /data/mysql/logs/slow.log
log_queries_not_using_indexes = 1
log_slow_admin_statements = 1
log_slow_slave_statements = 1
log_throttle_queries_not_using_indexes = 10
general_log_file = /data/mysql/logs/general.log
expire_logs_days = 8
long_query_time = 1
min_examined_row_limit = 100
​
​
########replication settings########
master_info_repository = TABLE
relay_log_info_repository = TABLE
log_bin = /data/mysql/binlogs/mysql-bin
sync_binlog = 0
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates=1
binlog_format = row 
relay_log = /data/mysql/relays/relay.log
relay_log_recovery = 1
​
​
########innodb settings########
innodb_page_size = 16384
innodb_buffer_pool_size = 1G
innodb_buffer_pool_instances = 8
innodb_lock_wait_timeout = 50
innodb_rollback_on_timeout = ON
innodb_buffer_pool_load_at_startup = 1
innodb_buffer_pool_dump_at_shutdown = 1
innodb_lru_scan_depth = 2000
innodb_io_capacity = 200
innodb_io_capacity_max = 800
innodb_flush_method = O_DIRECT
innodb_log_group_home_dir = /data/mysql/dbfiles
innodb_undo_directory = /data/mysql/dbfiles
innodb_undo_logs = 128
innodb_undo_tablespaces = 3
innodb_flush_neighbors = 1
innodb_log_file_size = 4G
innodb_log_buffer_size = 8M
innodb_purge_threads = 4
innodb_thread_concurrency = 64
innodb_print_all_deadlocks = 1
​
table_open_cache = 4096
open_files_limit = 8192
​
[mysqld-5.7]
innodb_buffer_pool_dump_pct = 40
innodb_page_cleaners = 4
innodb_undo_log_truncate = 1
innodb_max_undo_log_size = 2G
innodb_purge_rseg_truncate_frequency = 128
binlog_gtid_simple_recovery=1

2.5、配置环境变量

vim /etc/bashrc
...
​
# Mysql
export MYSQL_HOME=/usr/local/mysql-5.7.36
export PATH=$PATH:$MYSQL_HOME/bin

2.6、初始化数据库

mysqld --initialize --user=root --basedir=/usr/local/mysql-5.7.36 --datadir=/data/mysql/dbfiles

2.7、配置system管理文件

/usr/lib/systemd/system/mysqld.service

[Unit]
​
Description=mysql
After=network-online.target
​
[Service]
Type=forking
ExecStart=/usr/local/mysql-5.7.36/support-files/mysql.server start 
ExecStop=/usr/local/mysql-5.7.36/support-files/mysql.server stop
ExecReload=/usr/local/mysql-5.7.36/support-files/mysql.server restart
User=mysql
Group=mysql
Restart=always
RestartSec=5
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Delegate=yes
KillMode=process
​
[Install]
WantedBy=multi-user.target

2.8、服务启停

启动

systemctl start mysqld

停用

systemctl stop mysqld

开机启动

systemctl enable mysqld

三、xtrabackup操作数据库全量备份及还原

3.1、数据库备份

innobackuppex --defaults-file=/etc/my.cnf --user=root --password='DB_PASSWORD' -S SOCKET_FILE绝对路径 /data/db_backup

3.2、处理事务文件

innobackupex --apply-log /data/备份文件

3.3、修改my.cnf数据文件路径

[mysqld]
datadir = /data/mysql/NEW_DATA_DIR

3.4、授权

chown -R mysql. /data/mysql/NEW_DATA_DIR

3.5、重启数据库

systemctl restart mysqld

四、Mysql主从配置

4.1、创建主从同步用户并授权

create user slave@'%' identified by 'PASSWORD';
grant replication slave on *.* to slave@'%';

4.2、主从同步

查看logfile和logpos信息

[root@mysql-slave mysql]# cat dbfiles1/xtrabackup_info
uuid = e6748592-c0a4-11ec-bceb-080027211273
name =
tool_name = innobackupex
tool_command = --defaults-file=/etc/my.cnf --user=root --password=... -S /data/mysql/mysql.sock /data/db_backup/
tool_version = 2.4.21
ibbackup_version = 2.4.21
server_version = 5.7.36-log
start_time = 2022-04-20 20:24:59
end_time = 2022-04-20 20:25:01
lock_time = 1
binlog_pos = filename 'mysql-bin.000004', position '1146', GTID of the last change 'ff0ead77-bfe7-11ec-8af4-080027211273:1-8'
innodb_from_lsn = 0
innodb_to_lsn = 2662859
partial = N
incremental = N
format = file
compact = N
compressed = N
encrypted = N

主从同步

change master to master_host='192.168.56.30',master_user='slave',master_password='PASSWORD',master_log_file='LOGFILE',master_log_pos=POSID;

4.3、启动并检查主从状态

启动slave

start slave;

检查状态

show slave status\G;

五、xtrabackup操作数据库增量备份及还原

xtrabackup和mysql版本不匹配需要加:--no-server-version-check

5.1、操作数据全量备份

xtrabackup --backup -S /data/mysql/mysql.sock -uroot -p --target-dir=/data/db_backup/YYYYMMDD

5.2、按全量数据进行增量备份

xtrabackup --backup -S /data/mysql/mysql.sock -uroot -p --target-dir=/data/inc/inc-20220423 --incremental-basedir=/data/db_backup/20220423

5.3、处理全量数据的事务

xtrabackup --prepare --apply-log-only --target-dir=/data/db_backup/20220423

5.4、合并数据

xtrabackup --prepare --target-dir=/data/db_backup/20220423 --incremental-dir=/data/inc/inc-20220423

六、ansible-playbook自动化部署Mysql

6.1、ansible安装

  • 安装

curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py
python get-pip.py
python -m pip install --upgrade "pip < 21.0"
# pip安装ansible(国内如果安装太慢可以直接用pip阿里云加速)
pip install ansible -i https://mirrors.aliyun.com/pypi/simple/
  • 配置文件

/etc/ansible/ansible.cfg

[defaults]
gathering = smart
gather_timeout = 7
host_key_checking = False
private_role_vars = True
display_skipped_hosts = False
display_args_to_stdout = False
error_on_undefined_vars = True
system_warnings = False
deprecation_warnings = False
ssh_args = -o ControlMaster=auto -o ControlPersist=360s -o ServerAliveInterval=30 -o ServerAliveCountMax=2

6.2、配置ansible-playbook

目录

➜ playbook (master) ✔ tree .
.
├── mysql.yml
└── roles
    └── Mysql
        ├── files
        │   └── mysql-5.7.36-el7-x86_64.tar.gz
        ├── tasks
        │   └── main.yml
        └── templates
            ├── my5.7.36.cnf
            └── mysqld.service

6.2.1、创建playbook目录

---
- hosts: mysql
  vars:
    VERSION: 5.7.36
    PORT: 3306
    BASEDIR: "/data"
    # Set Memory 50%
    BUFFER_POOL_SIZE: 1
    ROOT_PASSWORD: 8ij34zziOaD0
  roles:
    - Mysql

6.2.2、创建role

mkdir -p role/mysql/{files,tasks,templates}

6.2.3、Tasks文件

- name: "Create User"
  shell: useradd --system mysql
  ignore_errors: true
​
- name: "Create dir"
  file: 
    path: "{{ item.dir }}"
    state: directory
    mode: 0755
    owner: "mysql"
    group: "mysql"
  with_items: 
    - { dir: "{{BASEDIR}}/mysql{{ PORT }}/binlogs" }
    - { dir: "{{BASEDIR}}/mysql{{ PORT }}/dbfiles" }
    - { dir: "{{BASEDIR}}/mysql{{ PORT }}/logs" }
    - { dir: "{{BASEDIR}}/mysql{{ PORT }}/relays" }
    - { dir: "{{BASEDIR}}/db_backup" }
​
- name: "Copy Mysql Package"
  unarchive:
    src: "mysql-{{ VERSION }}-el7-x86_64.tar.gz"
    dest: "/usr/local/"
​
- name: "Copy Mysql init sql"
  template:
    src: "init.sql"
    dest: "/opt/"
​
- name: "Rename dir name"
  shell: mv /usr/local/mysql-{{ VERSION }}-el7-x86_64 /usr/local/mysql-{{ VERSION }}
​
- name: "Copy Mysql Config"
  template:
    src: "my{{ VERSION }}.cnf"
    dest: "/etc/my.cnf"
​
- name: "Copy Mysql systemctl file"
  template:
    src: "mysqld.service"
    dest: "/etc/systemd/system/mysqld.service"
​
- name: "Init Mysql"
  shell: "/usr/local/mysql-{{ VERSION }}/bin/mysqld --initialize --user=root --basedir=/usr/local/mysql-{{ VERSION }} --datadir={{BASEDIR}}/mysql{{ PORT }}/dbfiles"
  

6.2.4、Templates 目录

  • My.cof文件

#MySQL_version = 5.7.36
[client]
socket=/data/mysql{{ PORT }}/mysql.sock
​
[mysqld]
port = {{ PORT }}
user = mysql
basedir = /usr/local/mysql-{{ VERSION }}
datadir = /data/mysql{{ PORT }}/dbfiles
tmpdir=/data/mysql{{ PORT }}
socket = /data/mysql{{ PORT }}/mysql.sock
pid-file=/data/mysql{{ PORT }}/mysql.pid
secure-file-priv=''
default_authentication_plugin = mysql_native_password
​
lower_case_table_names=1
​
server-id = {{ inventory_hostname.split('.')[-1] }}
transaction_isolation = READ-COMMITTED
autocommit = 1
character_set_server=utf8mb4
max_connections = 4000
max_connect_errors = 1844674407370954751
connect_timeout = 10
lock_wait_timeout=3600
thread_cache_size=256
#sql_mode = "NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO"
sql_mode = "NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER"
​
​
join_buffer_size = 4M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
max_allowed_packet = 16M
sort_buffer_size = 16M
​
########basic settings########
#skip-grant-tables
#secure_file_priv = /data/mysql{{ PORT }}/
skip_name_resolve = 1
explicit_defaults_for_timestamp = 1
tmp_table_size = 67108864
#interactive_timeout = 1800
#wait_timeout = 1800
​
​
########log settings########
log_error = /data/mysql{{ PORT }}/logs/mysql_err.log
log_timestamps = system
slow_query_log = 1
slow_query_log_file = /data/mysql{{ PORT }}/logs/slow.log
log_queries_not_using_indexes = 1
log_slow_admin_statements = 1
log_slow_slave_statements = 1
log_throttle_queries_not_using_indexes = 10
general_log_file = /data/mysql{{ PORT }}/logs/general.log
expire_logs_days = 8
long_query_time = 1
min_examined_row_limit = 100
​
​
########replication settings########
master_info_repository = TABLE
relay_log_info_repository = TABLE
log_bin = /data/mysql{{ PORT }}/binlogs/mysql-bin
sync_binlog = 0
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates=1
binlog_format = row 
relay_log = /data/mysql{{ PORT }}/relays/relay.log
relay_log_recovery = 1
​
​
########innodb settings########
innodb_page_size = 16384
innodb_buffer_pool_size = {{ BUFFER_POOL_SIZE }}G
innodb_buffer_pool_instances = 8
innodb_lock_wait_timeout = 50
innodb_rollback_on_timeout = ON
innodb_buffer_pool_load_at_startup = 1
innodb_buffer_pool_dump_at_shutdown = 1
innodb_lru_scan_depth = 2000
innodb_io_capacity = 200
innodb_io_capacity_max = 800
innodb_flush_method = O_DIRECT
innodb_log_group_home_dir = /data/mysql{{ PORT }}/dbfiles
innodb_undo_directory = /data/mysql{{ PORT }}/dbfiles
innodb_undo_logs = 128
innodb_undo_tablespaces = 3
innodb_flush_neighbors = 1
innodb_log_file_size = 4G
innodb_log_buffer_size = 8M
innodb_purge_threads = 4
innodb_thread_concurrency = 64
innodb_print_all_deadlocks = 1
​
table_open_cache = 4096
open_files_limit = 8192
​
[mysqld-5.7]
innodb_buffer_pool_dump_pct = 40
innodb_page_cleaners = 4
innodb_undo_log_truncate = 1
innodb_max_undo_log_size = 2G
innodb_purge_rseg_truncate_frequency = 128
binlog_gtid_simple_recovery=1
  • Mysqld.service模板

[Unit]
​
Description=mysql
After=network-online.target
​
[Service]
Type=forking
ExecStart=/usr/local/mysql-{{ VERSION }}/support-files/mysql.server start 
ExecStop=/usr/local/mysql-{{ VERSION }}/support-files/mysql.server stop
ExecReload=/usr/local/mysql-{{ VERSION }}/support-files/mysql.server restart
User=mysql
Group=mysql
Restart=always
RestartSec=5
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Delegate=yes
KillMode=process
​
[Install]
WantedBy=multi-user.target

七、prometheus监控Mysql

7.1、包下载

包目录:/data/softs

https://github.com/prometheus/mysqld_exporter/releases

7.2、Systemd服务管理

创建系统用户

  • useradd --system mysql_exporter

创建数据库监控用户

  • CREATE USER 'prometheus_monitor'@'%' IDENTIFIED BY 'PASSWORD' WITH MAX_USER_CONNECTIONS 3;
  • GRANT PROCESS, REPLICATION CLIENT, REPLICATION SLAVE, SELECT ON *.* TO 'prometheus_monitor'@'%';

启动

  • systemctl start mysqld_exporter.service

停止

  • systemctl stop mysqld_exporter.service

开机启动

  • systemctl enable mysqld_exporter.service

cat /usr/lib/systemd/system/mysqld_exporter.service
[Unit]
Description=mysqld_exporter
After=network.target
[Service]
Type=simple
User=mysql_exporter
Environment=DATA_SOURCE_NAME=MYSQL_USER:MYSQL_PASSWORD@(localhost:3306)/
ExecStart=/usr/local/sbin/mysqld_exporter --web.listen-address=0.0.0.0:9104 \
 --config.my-cnf /etc/my.cnf \
 --collect.slave_status \
 --collect.slave_hosts \
 --log.level=error \
 --collect.info_schema.processlist \
 --collect.info_schema.innodb_metrics \
 --collect.info_schema.innodb_tablespaces \
 --collect.info_schema.innodb_cmp \
 --collect.info_schema.innodb_cmpmem
Restart=on-failure
[Install]
WantedBy=multi-user.targe

 

 教学视频:

2022最新-mysql5.7.36二进制包部署_哔哩哔哩_bilibili
2022最新-02mysql5.7.36xtrabackup全备还原_哔哩哔哩_bilibili
2022最新-03mysql5.7.36主从配置_哔哩哔哩_bilibili
2022最新-04mysql5.7.36xtrabackup增量还原_哔哩哔哩_bilibili

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

Tale_G

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值