mysql oak_MySQL oak-online-alter-table工具使用初探

最新推荐文章于 2023-04-18 21:38:55 发布

算法艺术家

最新推荐文章于 2023-04-18 21:38:55 发布

阅读量470

点赞数

文章标签： mysql oak

本文链接：https://blog.csdn.net/weixin_28684825/article/details/113909353

版权

本文介绍了MySQL的Oak工具，用于在线DDL操作，例如非阻塞ALTER TABLE。详细步骤展示了如何添加列、索引等，强调了工具在不锁定表的情况下高效执行修改。建议在生产环境中使用。

摘要由CSDN通过智能技术生成

oak-online-alter-table小工具是用来实现MySQL Online DDL的

下载地址:https://code.google.com/archive/p/openarkkit/downloads

从下图中找到openark-kit-196-1.noarch.rpm

安装:

[root@idb4 ~]# rpm -ivh openark-kit-196-1.noarch.rpm

Preparing... ########################################### [100%]

1:openark-kit ########################################### [100%]

该工具提供了以下三种基本功能：

1、一个非阻塞ALTER TABLE操作，以下几种情况都是支持的

1)添加列 (新列必须有一个默认值)

2)删除列 (旧表必须有一个单列的唯一索引)

3)修改列 (改变字段类型,包括唯一键的列)

4)添加索引 (普通索引,唯一索引,全文索引)

5)删除索引(旧表必须有一个单列的唯一索引)

6) 修改表引擎:当处理非事务性引擎应该格外注意

7)添加外键约束

2、(可能会在未来版本不再支持):创建一个镜像表,与原始表同步,只要不发生如下操作:

1)对原始表ALTER TABLE操作

2)对原始表TRUNCATE操作

3)使用LOAD DATA INFILE向原始表导入数据

4)对原始表OPTIMIZE TABLE操作

3、一个空的ALTER,重建一个表:释放磁盘空间和重新组织表，相当于优化表。

检查表是否符合oak-online-alter-table的条件:

单列唯一索引(联合索引和联合主键是不可以的，因为会触发mysql的一个bug)

没有foreign key (没有外键,oak-online-alter-table对有外键的表是没有办法的)

没有定义触发器(有的话也要先删除)

线上环境的oak 命令用法:

oak-online-alter-table

具体帮助信息：

[root@idb4 data]# oak-online-alter-table --help

Usage: oak-online-alter-table [options]

Options:

-h, --help show this help message and exit

-u USER, --user=USER MySQL user

-H HOST, --host=HOST MySQL host (default: localhost)

-p PASSWORD, --password=PASSWORD

MySQL password

--ask-pass Prompt for password

-P PORT, --port=PORT TCP/IP port (default: 3306)

-S SOCKET, --socket=SOCKET

MySQL socket file. Only applies when host is localhost

--defaults-file=DEFAULTS_FILE

Read from MySQL configuration file. Overrides all

other options

-d DATABASE, --database=DATABASE

Database name (required unless table is fully

qualified)

-t TABLE, --table=TABLE

Table to alter (optionally fully qualified)

-g GHOST, --ghost=GHOST

Table name to serve as ghost. This table will be

created and synchronized with the original table

-a ALTER_STATEMENT, --alter=ALTER_STATEMENT

Comma delimited ALTER statement details, excluding the

'ALTER TABLE t' itself

-c CHUNK_SIZE, --chunk-size=CHUNK_SIZE

Number of rows to act on in chunks. Default: 1000

-l, --lock-chunks Use LOCK TABLES for each chunk

-N, --skip-binlog Disable binary logging

-r MAX_LOCK_RETRIES, --max-lock-retries=MAX_LOCK_RETRIES

Maximum times to retry on deadlock or

lock_wait_timeout. (default: 10; 0 is unlimited)

--skip-delete-pass Do not execute the DELETE data pass

--sleep=SLEEP_MILLIS Number of milliseconds to sleep between chunks.

Default: 0

--sleep-ratio=SLEEP_RATIO

Ratio of sleep time to execution time. Default: 0

--cleanup Remove custom triggers, ghost table from possible

previous runs

-v, --verbose Print user friendly messages

-q, --quiet Quiet mode, do not verbose

测试如下：

1、添加一个字段duansf

[root@idb4 ixinnuo_sfsj]# oak-online-alter-table -uroot -p123456 -S /tmp/mysql.sock --table=dsf_data --alter="ADD COLUMN duansf VARCHAR(64) DEFAULT ''"

-- ERROR: Errors found. Initiating cleanup

-- ERROR: No database specified. Specify with fully qualified table name or with -d or --database

报错，提示需要指定--database

加--database后成功执行:

[root@idb4 ixinnuo_sfsj]# oak-online-alter-table -uroot -p123456 -S /tmp/mysql.sock --database=ixinnuo_sfsj --table=dsf_data --alter="ADD COLUMN duansf VARCHAR(64) DEFAULT ''"

-- Connecting to MySQL

-- Table ixinnuo_sfsj.dsf_data is of engine innodb

-- Checking for UNIQUE columns on ixinnuo_sfsj.dsf_data, by which to chunk

-- Possible UNIQUE KEY column names in ixinnuo_sfsj.dsf_data:

-- - id,sh

-- Table ixinnuo_sfsj.__oak_dsf_data has been created

-- Table ixinnuo_sfsj.__oak_dsf_data has been altered

-- Checking for UNIQUE columns on ixinnuo_sfsj.__oak_dsf_data, by which to chunk

-- Possible UNIQUE KEY column names in ixinnuo_sfsj.__oak_dsf_data:

-- - id,sh

-- Checking for UNIQUE columns on ixinnuo_sfsj.dsf_data, by which to chunk

-- - Found following possible unique keys:

-- - id,sh (bigint)

-- Chosen unique key is 'id,sh'

-- Shared columns: status, update_time, kpjh, sh, month, create_time, fp_data, zfjh, id

-- Created AD trigger

-- Created AU trigger

-- Created AI trigger

-- Attempting to lock tables

-- Tables locked WRITE

-- id,sh (min, max) values: ([971717L, '33021155799011X'], [1174348L, '64010407380179X'])

-- Tables unlocked

-- - Reminder: altering ixinnuo_sfsj.dsf_data: ADD COLUMN duansf VARCHAR(64) ...

-- Copying range (971717,33021155799011X), (972716,440300683797687), progress: 0%

/usr/local/bin/oak-online-alter-table:84: Warning: Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT IGNORE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave.

num_affected_rows = cursor.execute(query)

/usr/local/bin/oak-online-alter-table:84: Warning: Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statements writing to a table with an auto-increment column after selecting from another table are unsafe because the order in which rows are retrieved determines what (if any) rows will be written. This order cannot be predicted and may differ on master and the slave.

num_affected_rows = cursor.execute(query)

-- Copying range (972716,440300683797687), (973716,340104688102768), progress: 0%

-- Copying range (973716,340104688102768), (974716,130302065712316), progress: 0%

-- Copying range (974716,130302065712316), (975716,91330201567021582Y), progress: 1%

-- Copying range (975716,91330201567021582Y), (976716,91330201567021582Y), progress: 1%

-- Copying range (976716,91330201567021582Y), (977716,91330201567021582Y), progress: 2%

-- Copying range (977716,91330201567021582Y), (978716,330226L66718333), progress: 2%

-- Copying range (978716,330226L66718333), (979716,34242119620528171701), progress: 3%

-- Copying range (979716,34242119620528171701), (980716,34242119620528171701), progress: 3%

-- Copying range (980716,34242119620528171701), (981716,330203074908847), progress: 4%

-- Copying range (981716,330203074908847), (982716,330226L17725262), progress: 4%

-- Copying range (982716,330226L17725262), (983716,91420100731061034W), progress: 5%

-- Copying range (983716,91420100731061034W), (984716,340104688102768), progress: 5%

-- Copying range (984716,340104688102768), (985716,440300683797687), progress: 6%

-- Copying range (985716,440300683797687), (986716,340181090787790), progress: 6%

-- Copying range (986716,340181090787790), (987716,91330201674719468Q), progress: 7%

-- Copying range (987716,91330201674719468Q), (988716,500108696565383), progress: 7%

-- Copying range (988716,500108696565383), (989716,440300590749985), progress: 8%

-- Copying range (989716,440300590749985), (990716,330281717286130), progress: 8%

-- Copying range (990716,330281717286130), (991716,130302065712316), progress: 9%

-- - Reminder: altering ixinnuo_sfsj.dsf_data: ADD COLUMN duansf VARCHAR(64) ...

-- Copying range (991716,130302065712316), (992716,131102063134364), progress: 9%

-- Copying range (992716,131102063134364), (993716,330204756267282), progress: 10%

-- Copying range (993716,330204756267282), (994716,91110105318223036H), progress: 10%

-- Copying range (994716,91110105318223036H), (995716,91420117572023195P), progress: 11%

-- Copying range (995716,91420117572023195P), (996716,91330204MA2824NX66), progress: 11%

-- Copying range (996716,91330204MA2824NX66), (997716,913205065617808877), progress: 12%

-- Copying range (997716,913205065617808877), (998716,500903765946981), progress: 12%

-- Copying range (998716,500903765946981), (999716,440301724700897), progress: 13%

-- Copying range (999716,440301724700897), (1000716,330282587473615), progress: 13%

-- Copying range (1000716,330282587473615), (1001716,913302011447225710), progress: 14%

-- Copying range (1001716,913302011447225710), (1002716,91330203573663733B), progress: 14%

-- Copying range (1002716,91330203573663733B), (1003716,91330204053841348X), progress: 15%

-- Copying range (1003716,91330204053841348X), (1004716,91330204053841348X), progress: 15%

-- Copying range (1004716,91330204053841348X), (1005716,310228630999533), progress: 16%

-- Copying range (1005716,310228630999533), (1006716,310228630999533), progress: 16%

-- Copying range (1006716,310228630999533), (1007716,91310112679338381W), progress: 17%

-- Copying range (1007716,91310112679338381W), (1008716,330227563850615), progress: 17%

-- Copying range (1008716,330227563850615), (1009716,330227563850615), progress: 18%

-- Copying range (1009716,330227563850615), (1010716,330227563850615), progress: 18%

-- Copying range (1010716,330227563850615), (1011716,330203698235871), progress: 19%

-- - Reminder: altering ixinnuo_sfsj.dsf_data: ADD COLUMN duansf VARCHAR(64) ...

-- Copying range (1011716,330203698235871), (1012716,9133028214483969XM), progress: 19%

-- Copying range (1012716,9133028214483969XM), (1013716,440301772706775), progress: 20%

-- Copying range (1013716,440301772706775), (1014716,9133028214483969XM), progress: 20%

-- Copying range (1014716,9133028214483969XM), (1015716,91330226747385420R), progress: 21%

-- Copying range (1015716,91330226747385420R), (1016716,330203665593904), progress: 21%

-- Copying range (1016716,330203665593904), (1017716,131102063134364), progress: 22%

-- Copying range (1017716,131102063134364), (1018716,330226758887514), progress: 22%

-- Copying range (1018716,330226758887514), (1019716,330226758887514), progress: 23%

-- Copying range (1019716,330226758887514), (1020716,91330226062904195F), progress: 23%

..........................................................................................

-- Deleting range (1158716,330281L41374386), (1159716,330281L41374386), progress: 92%

-- Deleting range (1159716,330281L41374386), (1160716,510107743634485), progress: 92%

-- Deleting range (1160716,510107743634485), (1161716,110108569534285), progress: 93%

-- Deleting range (1161716,110108569534285), (1162716,91330212681091461J), progress: 93%

-- Deleting range (1162716,91330212681091461J), (1163716,91330212681091461J), progress: 94%

-- Deleting range (1163716,91330212681091461J), (1164716,91330206695072544C), progress: 94%

-- Deleting range (1164716,91330206695072544C), (1165716,91330206695072544C), progress: 95%

-- Deleting range (1165716,91330206695072544C), (1166716,330282780442490), progress: 95%

-- Deleting range (1166716,330282780442490), (1167716,330204736982430), progress: 96%

-- Deleting range (1167716,330204736982430), (1168716,64010407380179X), progress: 96%

-- Deleting range (1168716,64010407380179X), (1169716,91330281684260355F), progress: 97%

-- Deleting range (1169716,91330281684260355F), (1170716,915002367116623424), progress: 97%

-- Deleting range (1170716,915002367116623424), (1171716,91310112679338381W), progress: 98%

-- - Reminder: altering ixinnuo_sfsj.dsf_data: ADD COLUMN duansf VARCHAR(64) ...

-- Deleting range (1171716,91310112679338381W), (1172716,91330204726429965L), progress: 98%

-- Deleting range (1172716,91330204726429965L), (1173716,91510113732356280Q), progress: 99%

-- Deleting range (1173716,91510113732356280Q), (1174348,64010407380179X), progress: 99%

-- Deleting range 100% complete. Number of rows: 0

-- Table ixinnuo_sfsj.dsf_data has been renamed to ixinnuo_sfsj.__arc_dsf_data,

-- and table ixinnuo_sfsj.__oak_dsf_data has been renamed to ixinnuo_sfsj.dsf_data

-- Table ixinnuo_sfsj.__arc_dsf_data was found and dropped

-- ALTER TABLE completed

查看新增加的列:

mysql> show columns from dsf_data;

+-------------+--------------+------+-----+---------+----------------+

+-------------+--------------+------+-----+---------+----------------+

| id | bigint(20) | NO | PRI | NULL | auto_increment |

| SH | varchar(32) | NO | PRI | | |

+-------------+--------------+------+-----+---------+----------------+

10 rows in set (0.00 sec)

在新增加的列字段duansf上添加普通索引：

[root@idb4 ixinnuo_sfsj]# oak-online-alter-table -uroot -p123456 -S /tmp/mysql.sock --database=ixinnuo_sfsj --table=dsf_data --alter="ADD KEY(duansf)"

-- Connecting to MySQL

-- Table ixinnuo_sfsj.dsf_data is of engine innodb

-- Checking for UNIQUE columns on ixinnuo_sfsj.dsf_data, by which to chunk

-- Possible UNIQUE KEY column names in ixinnuo_sfsj.dsf_data:

-- - id,sh

-- Table ixinnuo_sfsj.__oak_dsf_data has been created

-- Table ixinnuo_sfsj.__oak_dsf_data has been altered

-- Checking for UNIQUE columns on ixinnuo_sfsj.__oak_dsf_data, by which to chunk

-- Possible UNIQUE KEY column names in ixinnuo_sfsj.__oak_dsf_data:

-- - id,sh

-- Checking for UNIQUE columns on ixinnuo_sfsj.dsf_data, by which to chunk

-- - Found following possible unique keys:

-- - id,sh (bigint)

-- Chosen unique key is 'id,sh'

-- Shared columns: status, update_time, kpjh, duansf, month, sh, create_time, fp_data, zfjh, id

-- Created AD trigger

-- Created AU trigger

-- Created AI trigger

-- Attempting to lock tables

-- Tables locked WRITE

-- id,sh (min, max) values: ([971717L, '33021155799011X'], [1174348L, '64010407380179X'])

-- Tables unlocked

-- - Reminder: altering ixinnuo_sfsj.dsf_data: ADD KEY(duansf)...