MySQL大版本升级

目录

1.背景

2.环境

3.安装mysql-shell并检查配置文件

4.根据mysqlsh检查项提示修改项

5.关闭数据库

6.防止升级失败，做冷备

7.解压MySQL8.0安装包

8.解压安装包到旧版本的路径上

9.启动数据库

10.注意事项

11.回退

---

1.背景

      因MySQL 5.7 版本将于 2023 年 10 月EOL，AWS RDS MySQL 5.7 也将在 10 月停止该版本的支持和服务；客户需要将数据库从5.7升级到8.0，客户领导经过综合考虑将数据库升级为8.0.33\34（其中很大部分原因都是因为漏洞扫描）

     

   PS：我们升级数据库的时候需要比最新版本低1-2个版本，，因为新版本一般bug比较多，让别人替咱们踩踩坑；其实大部分漏洞扫描升级到8.0.29就解决了大部分高危漏洞啦，但是为什么还是要升级到8.0.33\34呢，因为从MySOL8.0.29开始执行alter table add/drop 默认的ALGORITHM是INSTANT算法。而PXB（xtrabackup）官网从8.0.29-8.0.31不支持instant算法。

    难道这个bug没有解决办法吗？当然也有解决办法，首先需要说明的是这个bug并没有被PXB官方明确升级为bug，但是在xtrabackup8.0.32中修复啦。

    临时解决办法：

    SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLES WHERE TOTAL_ROW_VERSIONS > 0;（错误信息中也有提示）

然后根据结果：OPTIMIZE TABLE TABLENAME

执行完后，可进行备份

官网描述：Error Message: Found tables with row versions due to INSTANT ADD/DROP columns - Percona XtraBackup

    但是如果遇到对操作比较严格的公司来说，我们不能每次执行完alter table add/drop都要执行optimize table tablename，跑的快的可以去试试。

2.环境

IP	角色	数据库版本	升级后版本
10.186.65.52	master	MySQL5.7.43	MySQL8.0.34
10.186.65.2	slave	MySQL5.7.43	MySQL8.0.34

主从做升级的时候要先升级从库，因为MySQL是向下兼容的。

3.安装mysql-shell并检查配置文件

## 下载地址：https://dev.mysql.com/downloads/repo/yum/
   
[root@localhost ~]# wget https://dev.mysql.com/get/mysql80-community-release-el7-10.noarch.rpm
[root@localhost ~]# rpm -Uvh mysql80-community-release-el7-10.noarch.rpm
warning: mysql80-community-release-el7-10.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 3a79bd29: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql80-community-release-el7-10 ################################# [100%]
[root@localhost ~]# yum -y install mysql-shell

4.执行升级检查命令

MySQL 5.7升级或迁移至MySQL 8.0时，由于一些功能或参数被废除，或者一些新的限制条件，因此，必须要做升级前的检查。MySQL shell提供了升级检查工具check-for-server-upgrade()，该工具可将一些兼容性问题以及其他错误或警告信息以Errors、Warnings和Notices的方式打印出来，方便我们了解有哪些必须在升级前修改具体检查项。

mysqlsh -- util check-for-server-upgrade { --user=root --socket=/opt/mysql/data/3306/mysqld.sock } --target-version=8.0.34  --config-path=/opt/mysql/etc/3306/my.cnf
The MySQL server at /opt%2Fmysql%2Fdata%2F3306%2Fmysqld.sock, version 5.7.43-log
- MySQL Community Server (GPL), will now be checked for compatibility issues
for upgrade to MySQL 8.0.34...
  
1) Usage of old temporal type
  No issues found
  
2) MySQL 8.0 syntax check for routine-like objects
  No issues found
  
3) Usage of db objects with names conflicting with new reserved keywords
  No issues found
  
4) Usage of utf8mb3 charset
  Warning: The following objects use the utf8mb3 character set. It is
    recommended to convert them to use utf8mb4 instead, for improved Unicode
    support.
  More information:
    https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8mb3.html
  
  slow_log.instances.name - column's default character set: utf8
  slow_log.instances.dsn - column's default character set: utf8
  slow_log.instances.distro - column's default character set: utf8
  slow_log.instances.version - column's default character set: utf8
  slow_log.query_classes.checksum - column's default character set: utf8
  slow_log.query_classes.abstract - column's default character set: utf8
  slow_log.query_classes.fingerprint - column's default character set: utf8
  slow_log.query_classes.tables - column's default character set: utf8
  slow_log.query_classes.status - column's default character set: utf8
  slow_log.query_examples.db - column's default character set: utf8
  slow_log.query_examples.query - column's default character set: utf8
  
5) Table names in the mysql schema conflicting with new tables in 8.0
  No issues found
  
6) Partitioned tables using engines with non native partitioning
  No issues found
  
7) Foreign key constraint names longer than 64 characters
  No issues found
  
8) Usage of obsolete MAXDB sql_mode flag
  No issues found
  
9) Usage of obsolete sql_mode flags
  Notice: The following DB objects have obsolete options persisted for
    sql_mode, which will be cleared during upgrade to 8.0.
  More information:
    https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html#mysql-nutshell-removals
  
  global system variable sql_mode - defined using obsolete NO_AUTO_CREATE_USER
    option
  
10) ENUM/SET column definitions containing elements longer than 255 characters
  No issues found
  
11) Usage of partitioned tables in shared tablespaces
  No issues found
  
12) Circular directory references in tablespace data file paths
  No issues found
  
13) Usage of removed functions
  No issues found
  
14) Usage of removed GROUP BY ASC/DESC syntax
  No issues found
  
15) Removed system variables for error logging to the system log configuration
  No issues found
  
16) Removed system variables
  Error: Following system variables that were detected as being used will be
    removed. Please update your system to not rely on them before the upgrade.
  More information:
    https://dev.mysql.com/doc/refman/8.0/en/added-deprecated-removed.html#optvars-removed
  
  query_cache_size - is set and will be removed
  query_cache_type - is set and will be removed
  
17) System variables with new default values
  Warning: Following system variables that are not defined in your
    configuration file will have new default values. Please review if you rely on
    their current values and if so define them before performing upgrade.
  More information:
    https://mysqlserverteam.com/new-defaults-in-mysql-8-0/
  
  back_log - default value will change
  collation_server - default value will change from latin1_swedish_ci to
    utf8mb4_0900_ai_ci
  event_scheduler - default value will change from OFF to ON
  explicit_defaults_for_timestamp - default value will change from OFF to ON
  innodb_autoinc_lock_mode - default value will change from 1 (consecutive) to
    2 (interleaved)
  innodb_flush_neighbors - default value will change from 1 (enable) to 0
    (disable)
  innodb_max_dirty_pages_pct_lwm - default value will change from_0 (%) to 10
    (%)
  innodb_undo_tablespaces - default value will change from 0 to 2
  log_error_verbosity - default value will change from 3 (Notes) to 2 (Warning)
  max_error_count - default value will change from 64 to 1024
  optimizer_trace_max_mem_size - default value will change from 16KB to 1MB
  performance_schema_consumer_events_transactions_current - default value will
    change from OFF to ON
  performance_schema_consumer_events_transactions_history - default value will
    change from OFF to ON
  slave_rows_search_algorithms - default value will change from 'INDEX_SCAN,
    TABLE_SCAN' to 'INDEX_SCAN, HASH_SCAN'
  transaction_write_set_extraction - default value will change from OFF to
    XXHASH64
  
18) Zero Date, Datetime, and Timestamp values
  Warning: By default zero date/datetime/timestamp values are no longer allowed
    in MySQL, as of 5.7.8 NO_ZERO_IN_DATE and NO_ZERO_DATE are included in
    SQL_MODE by default. These modes should be used with strict mode as they will
    be merged with strict mode in a future release. If you do not include these
    modes in your SQL_MODE setting, you are able to insert
    date/datetime/timestamp values that contain zeros. It is strongly advised to
    replace zero values with valid ones, as they may not work correctly in the
    future.
  More information:
    https://lefred.be/content/mysql-8-0-and-wrong-dates/
  
  global.sql_mode - does not contain either NO_ZERO_DATE or NO_ZERO_IN_DATE
    which allows insertion of zero dates
  session.sql_mode -  of 20 session(s) does not contain either NO_ZERO_DATE or
    NO_ZERO_IN_DATE which allows insertion of zero dates
  
19) Schema inconsistencies resulting from file removal or corruption
  No issues found
  
20) Tables recognized by InnoDB that belong to a different engine
  No issues found
  
21) Issues reported by 'check table x for upgrade' command
  No issues found
  
22) New default authentication plugin considerations
  Warning: The new default authentication plugin 'caching_sha2_password' offers
    more secure password hashing than previously used 'mysql_native_password'
    (and consequent improved client connection authentication). However, it also
    has compatibility implications that may affect existing MySQL installations.
    If your MySQL installation must serve pre-8.0 clients and you encounter
    compatibility issues after upgrading, the simplest way to address those
    issues is to reconfigure the server to revert to the previous default
    authentication plugin (mysql_native_password). For example, use these lines
    in the server option file:
      
    [mysqld]
    default_authentication_plugin=mysql_native_password
      
    However, the setting should be viewed as temporary, not as a long term or
    permanent solution, because it causes new accounts created with the setting
    in effect to forego the improved authentication security.
    If you are using replication please take time to understand how the
    authentication plugin changes may impact you.
  More information:
    https://dev.mysql.com/doc/refman/8.0/en/upgrading-from-previous-series.html#upgrade-caching-sha2-password-compatibility-issues
    https://dev.mysql.com/doc/refman/8.0/en/upgrading-from-previous-series.html#upgrade-caching-sha2-password-replication
  
23) Columns which cannot have default values
  No issues found
  
24) Check for invalid table names and schema names used in 5.7
  No issues found
  
25) Check for orphaned routines in 5.7
  No issues found
  
26) Check for deprecated usage of single dollar signs in object names
  No issues found
  
27) Check for indexes that are too large to work on higher versions of MySQL
Server than 5.7
  No issues found
  
28) Check for deprecated '.<table>' syntax used in routines.
  No issues found
  
Errors:   2
Warnings: 29
Notices:  1
  
ERROR: 2 errors were found. Please correct these issues before upgrading to avoid compatibility issues.

5.根据mysqlsh检查项提示修改项

query_cache_size，query_cache_type在MySQL8.0.3中删除
expire_logs_days在MySQL8.0.3中已弃用 已使用binlog_expire_logs_seconds 在这么多秒后清除二进制日志
   
5.7和8.0的默认身份证插件发生变化，为了防止升级后的兼容性问题，临时解决办法是：
default_authentication_plugin=mysql_native_password
   
sql_mode使用过时参数NO_AUTO_CREATE_USER
   
在配置文件中修改以上信息
   
根据检查修改配置文件，然后再做一次升级检查看看存在什么问题

6.关闭数据库

设置参数用于在关闭数据库后将平时数据库中所有的数据，日志刷到磁盘，关闭快速关机
mysql> set global innodb_fast_shutdown=0;
Query OK, 0 rows affected (0.00 sec)
   
mysql> select @@innodb_fast_shutdown;
+------------------------+
| @@innodb_fast_shutdown |
+------------------------+
|                      0 |
+------------------------+
1 row in set (0.00 sec)
   
关闭数据库
[root@localhost ~]# systemctl stop mysqld_3306

7.防止升级失败，做冷备

备份项：data目录,undo log,redo log

[root@localhost ~]# cp -rp /opt/mysql/data/3306 /opt/mysql/data/3306_bak
[root@localhost ~]# cp -rp /opt/mysql/log/redolog/3306 /opt/mysql/log/redolog/3306_bak

Ps:我的undo log在data目录下面，像我这种手残党，总是喜欢做一些匪夷所思的操作，所以我还会进行一次全备。

8.解压MySQL8.0安装包

[root@localhost ~]# tar xvf mysql-8.0.34-linux-glibc2.12-x86_64.tar.xz

9.解压安装包到旧版本的路径上

[root@localhost ~]# cd /opt/mysql/base
   
[root@localhost base]# cp -r /root/mysql-8.0.34-linux-glibc2.12-x86_64/ /opt//mysql/base/8.0.34
[root@localhost base]# chown -R actiontech-universe:actiontech ./

10.启动数据库

[root@localhost ~]# systemctl start mysqld_3306
   
* 查看数据库版本号
mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.34    | 
+-----------+
1 row in set (0.00 sec)
   
* 启动复制
mysql> start slave;
   
查看复制是否正常
mysql> show slave status
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 10.186.65.52
                  Master_User: universe_op
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000009
          Read_Master_Log_Pos: 222211681
               Relay_Log_File: mysql-relay.000006
                Relay_Log_Pos: 58459
        Relay_Master_Log_File: mysql-bin.000009
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 222211681
              Relay_Log_Space: 58665
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 621513
                  Master_UUID: c8bdfe0c-2b68-11ee-8d56-02000aba413c
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 1
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set: c8bdfe0c-2b68-11ee-8d56-02000aba413c:437404-437420
            Executed_Gtid_Set: 69e72c25-56b9-11ee-a55c-02000aba4177:1,
a0d6a728-56b7-11ee-983e-02000aba4177:1-24,
c8bdfe0c-2b68-11ee-8d56-02000aba413c:1-437420
                Auto_Position: 1
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec

从库升级完啦，将从库提升为主，连接业务，然后升级旧主，还是上面的步骤。

11.注意事项

     MySQL 5.7升级至MySQL 8.0，需要注意的事项有：

1. 一定要先做升级前检查；

2. 安装MySQL 8.0的实例，要选择8.0的配置文件，同时需要根据之前5.7的参数配置作适配；

3. 迁移到MySQL 8.0之后，程序连接建议使用mysql-connector-java-8.x.jar去连接MySQL 8.0的版本；

4. 系统表的升级，在MySQL8.0.16版本之前，需要手动的执行mysql_upgrade来完成系统表的升级，在MySQL8.0.16版本之后是由mysqld来完成；

5. 事先准备8.0.34安装包和MySQL Shell安装包；

6. 在复制架构中，先升级从库，后升级主库。

PS：mysql_upgrade手工执行命令：mysql_upgrade -u root -p

mysql_upgrade检查所有数据库中的所有表是否与当前版本的MySQL不兼容。mysql_upgrade还会升级mysql系统数据库，以便您可以利用新的权限或功能。

12.回退

   如果要回退的话只需要将新版本的base文件和数据文件移动到其他地方，将之前备份的MySQL 5.7的base路径和数据文件拷贝回原始位置，然后重新启动。