目录
一、环境信息
名称 | 值 |
CPU | Intel(R) Core(TM) i5-1035G1 CPU @ 1.00GHz |
操作系统 | CentOS Linux release 7.9.2009 (Core) |
内存 | 3G |
逻辑核数 | 2 |
Gbase8a版本 | 8.6.2-R43.34.27468a27 |
EasyClean版本 | V1.1 |
二、简述
工作和兴趣相结合的产物,既能更好的完成工作,也能看看自己的学习情况如何,无论如何,大家一起加油。
三、升级点
序号 | 功能 | 备注 |
1 | 避免对于视图的操作 | 视图不需要做清理空洞率操作。 |
2 | 清理空洞率步骤函数的优化 | 避免某些特殊场景下的异常,例如SQL或执行者进程被强杀。 |
3 | 对于消息队列中的任务进行限制 | 避免执行者进程执行任务过慢,导致消息队列中的任务积压,致使达到消息队列最大字节数限制。 |
四、支持功能
序号 | 功能 | 备注 |
1 | 多进程执行任务 | |
2 | 空洞率清理 | |
3 | 自定义配置 | 后续会细讲参数。 |
4 | SQL重试功能 | 如果sql执行出错,会尝试3次。 |
五、空洞率
大家可以参考之前的博客《南大通用数据库-Gbase-8a-学习-33-空洞率查询与解决方法》。
六、工具流程图
1、流程描述
(1)管理者进程检查传入参数是否正确。
(2)管理者进程启动会去Gbase8a中检索需要清理空洞率的表。
(3)管理者进程获取环境变量和创建消息队列。
(4)管理者进程创建多个执行者进程。
(5)管理者进程向消息队列发送消息,当消息队列中的任务到达100件时,获取消息队列状态,管理者进程休眠5s,继续获取消息队列中的任务数。
(6)执行者进程检查传入参数是否正确。
(7)执行者连接数据库和获取环境变量、连接消息队列。
(8)执行者从消息队列中接收消息。
(9)执行者进程操作数据库清理空洞率。
(10)管理者进程发送完所有清理的表,向消息队列发送完成任务消息。
(11)执行者进程接收到完成任务消息,清理申请的资源。
(12)管理者进程回收所有执行者进程的PCB资源。
(13)管理者进程关闭消息队列。
(14)管理者进程清理申请的资源。
2、注意点
如果大家遇到某些大表清理空洞率较慢,想终止此表的清理,推荐方法如下:
(1)方法一
登录Gbase8a,使用kill命令强杀SQL,需要杀三次相同的SQL,因为工具有错误重试机制。
(2)方法二
操作系统层杀死执行SQL的执行者进程,这样会影响清理空洞率的效率,个人更推荐方法一。
不建议大家杀死管理者进程,这样会导致执行者进程的PCB资源无法回收,变成僵尸进程,虽然操作系统的init进程会接管僵尸进程和释放其资源,但毕竟不由我们程序自己控制,不保险。还有可能导致消息队列没有清理,出现任务残留,影响下一次程序运行。
七、安装包下载地址
已经放到开头啦,欢迎大家测试使用。电脑端才可以看见安装包。
八、参数介绍
1、命令模板
[gbase@czg2 Exec]$ ./Manager 'DbHost;DbUser;DbPwd;DbName;DbPort;DbCharset;ChdProcessNum;TargetDb;VoidRate;CleanTabNum;'
2、命令样例
[gbase@czg2 Exec]$ ./Manager '192.168.142.12;czg;qwer1234;gbase;5258;utf8;3;zxj;0;2;'
3、参数表格
序号 | 参数 | 备注 |
1 | DbHost | 连接源端数据库IP。 |
2 | DbUser | 连接源端数据库用户。 |
3 | DbPwd | 连接源端数据库用户密码。 |
4 | DbName | 连接源端数据库。 |
5 | DbPort | 连接源端数据库端口号。 |
6 | DbCharset | 连接源端数据库的字符集。 |
7 | ChdProcessNum | 启动的子进程数。 |
8 | TargetDb | 需要清理空洞率的数据库。 |
9 | VoidRate | 空洞率到达此值时进行清理,1-100。 |
10 | CleanTabNum | 清理TargetDb下空洞率超过VoidRate的表的个数。如果是0,表示无限制。 |
九、安装步骤
大家可以看README的内容,其实是一样的。
下面的配置大家根据实际情况来,我这边只是给一个例子。
1、配置环境变量
/home/gbase/.bashrc中添加如下
export CLEAN_VOID_RATE_TOOL_HOME=/home/gbase/EasyClean/
export LD_LIBRARY_PATH=$CLEAN_VOID_RATE_TOOL_HOME/Libs:$LD_LIBRARY_PATH
2、生效环境变量
source /home/gbase/.bashrc
3、检验动态链接是否正常
[gbase@czg0 Exec]$ ldd Manager
linux-vdso.so.1 => (0x00007ffe315b5000)
libPublicFunction.so => /home/gbase/EasyClean//Libs/libPublicFunction.so (0x00007f54c1a1b000)
libLog.so => /home/gbase/EasyClean//Libs/libLog.so (0x00007f54c1817000)
libGbase8aOperate.so => /home/gbase/EasyClean//Libs/libGbase8aOperate.so (0x00007f54c160a000)
libgbase.so.16 => /home/gbase/EasyClean//Libs/libgbase.so.16 (0x00007f54c114a000)
libSqQueue.so => /home/gbase/EasyClean//Libs/libSqQueue.so (0x00007f54c0f45000)
libDataConvertion.so => /home/gbase/EasyClean//Libs/libDataConvertion.so (0x00007f54c0d42000)
libProcess.so => /home/gbase/EasyClean//Libs/libProcess.so (0x00007f54c0b3e000)
libFileOperate.so => /home/gbase/EasyClean//Libs/libFileOperate.so (0x00007f54c0938000)
libMyHashTable.so => /home/gbase/EasyClean//Libs/libMyHashTable.so (0x00007f54c0734000)
libc.so.6 => /lib64/libc.so.6 (0x00007f54c0366000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f54c014a000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f54bff46000)
libm.so.6 => /lib64/libm.so.6 (0x00007f54bfc44000)
/lib64/ld-linux-x86-64.so.2 (0x00007f54c1c1e000)
[gbase@czg0 Exec]$ ldd Executor
linux-vdso.so.1 => (0x00007ffddffde000)
libPublicFunction.so => /home/gbase/EasyClean//Libs/libPublicFunction.so (0x00007f7563dff000)
libLog.so => /home/gbase/EasyClean//Libs/libLog.so (0x00007f7563bfb000)
libGbase8aOperate.so => /home/gbase/EasyClean//Libs/libGbase8aOperate.so (0x00007f75639ee000)
libgbase.so.16 => /home/gbase/EasyClean//Libs/libgbase.so.16 (0x00007f756352e000)
libSqQueue.so => /home/gbase/EasyClean//Libs/libSqQueue.so (0x00007f7563329000)
libDataConvertion.so => /home/gbase/EasyClean//Libs/libDataConvertion.so (0x00007f7563126000)
libProcess.so => /home/gbase/EasyClean//Libs/libProcess.so (0x00007f7562f22000)
libFileOperate.so => /home/gbase/EasyClean//Libs/libFileOperate.so (0x00007f7562d1c000)
libMyHashTable.so => /home/gbase/EasyClean//Libs/libMyHashTable.so (0x00007f7562b18000)
libc.so.6 => /lib64/libc.so.6 (0x00007f756274a000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f756252e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f756232a000)
libm.so.6 => /lib64/libm.so.6 (0x00007f7562028000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7564002000)
如果有动态库没有找到,就要看看环境变量是否配置正确或是否生效。
十、运行效果
[gbase@czg2 Exec]$ ./Manager '192.168.142.12;czg;qwer1234;gbase;5258;utf8;2;bd_db_apblc;0;0;'
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Info ]-EasyClean-V1.1-Manager.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Init SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Init SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-InitInArgvSt : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Split Str To Queue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 10.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MyStrToLong : OK, Str : 5258, Base : 10, RetVal : 5258.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MyStrToLong : OK, Str : 2, Base : 10, RetVal : 2.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MyStrToLong : OK, Str : 0, Base : 10, RetVal : 0.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Check Input Arg : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Connect Info : OK , DbHost : '192.168.142.12', DbUser : 'czg', DbName : 'gbase', DbPort : 5258.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-client character : utf8
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Init DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Sql Query : SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_SCHEMA =
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect -1 Rows.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name : TABLE_NAME
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name : TABLE_NAME
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Table Name : TABLES
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Db Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Default Val : (null)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Define Column Len : 192
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Max Column Len : 29
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name Len : 10
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name Len : 10
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Charset : 33
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Flags : 1 , Info : not null
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Type : 253 , Info : VARCHAR
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Fetch DQL Data : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Read Db All Table : OK
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Clear SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Init DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Sql Query : SELECT COUNT(*) FROM PERFORMANCE_SCHEMA.TABLES WHERE TABLE_SCHEMA='bd_db_apblc' AND TABLE_NAME='gbas
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect -1 Rows.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name : COUNT(*)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Db Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Default Val : (null)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Define Column Len : 21
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Max Column Len : 1
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name Len : 8
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name Len : 0
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Charset : 63
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Flags : 32897 , Info : Unkown Flags!!!
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Type : 8 , Info : BIGINT
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Fetch DQL Data : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Destroy DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Enter SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Init DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Sql Query : SELECT COUNT(*) FROM PERFORMANCE_SCHEMA.TABLES WHERE TABLE_SCHEMA='bd_db_apblc' AND TABLE_NAME='sg_t
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect -1 Rows.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name : COUNT(*)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Db Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Default Val : (null)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Define Column Len : 21
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Max Column Len : 1
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name Len : 8
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name Len : 0
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Charset : 63
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Flags : 32897 , Info : Unkown Flags!!!
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Type : 8 , Info : BIGINT
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Fetch DQL Data : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Destroy DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Enter SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Init DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Sql Query : SELECT COUNT(*) FROM PERFORMANCE_SCHEMA.TABLES WHERE TABLE_SCHEMA='bd_db_apblc' AND TABLE_NAME='sg_t
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect -1 Rows.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name : COUNT(*)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Table Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Db Name :
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Default Val : (null)
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Define Column Len : 21
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Max Column Len : 1
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Alias Name Len : 8
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Column Name Len : 0
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Charset : 63
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Flags : 32897 , Info : Unkown Flags!!!
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Type : 8 , Info : BIGINT
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Fetch DQL Data : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Destroy DqlResult : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Enter SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-ChkDbLvTabVoidRate : OK, DbName : bd_db_apblc, ClearTabNums : 0, VoidRate : 0.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 3.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Get Os Env : OK, OsEnvName : CLEAN_VOID_RATE_TOOL_HOME, RetVal : /opt/Developer/CleanVoidRate/.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-File2Key : OK, RetKey : 51f5.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MessageGet : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MyFork : OK, Pid : 43996.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Enter SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140539952031552]-[Debug]-MyFork : OK, Pid : 0.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MyFork : OK, Pid : 43997.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Enter SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-SqQueue Data :
Data : [ 43996 ,43997 ]
FrontIndex : 0
RearIndex : 2
SqQueueLen : 2
SqQueueMaxLen : 20
Flag : INT_TYPE_FLAG
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Printf InArgvSt :
SourceHost : 192.168.142.12
SourceUser : czg
SourcePwd : qwer1234
SourceDbName : gbase
SourcePort : 5258
SourceCharset : utf8
TargetDb : bd_db_apblc
VoidRate : 0
CleanTabNum : 0
ToolPath : /opt/Developer/CleanVoidRate/
ExecutorPath : /opt/Developer/CleanVoidRate/Exec/Executor
PublicStr : /opt/Developer/CleanVoidRate/Exec/File2Key.Data
ChdProcessNum : 2
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-SqQueue Data :
Data : [ 'gbase8a_i2b_config_table' ,'sg_t_loadconfig_incr_20230801' ,'sg_t_loadconfig_incr_odm' ]
FrontIndex : 0
RearIndex : 3
SqQueueLen : 3
SqQueueMaxLen : 1024
Flag : STRING_TYPE_FLAG
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 3.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Leave SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MessageSend : OK, MsgSize : 24, MsgType : 1, MsgInfo : 'gbase8a_i2b_config_table'.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 2.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Leave SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MessageSend : OK, MsgSize : 29, MsgType : 1, MsgInfo : 'sg_t_loadconfig_incr_20230801'.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 1.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Leave SqQueue : OK.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MessageSend : OK, MsgSize : 24, MsgType : 1, MsgInfo : 'sg_t_loadconfig_incr_odm'.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 0.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MessageSend : OK, MsgSize : 6, MsgType : 1, MsgInfo : 'Finish'.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-MessageSend : OK, MsgSize : 6, MsgType : 1, MsgInfo : 'Finish'.
2024-01-03 09:49:20-P[43992]-T[140539952031552]-[Debug]-Leave SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Info ]-EasyClean-V1.1-Executor.
2024-01-03 09:49:20-P[43997]-T[140539952031552]-[Debug]-MyFork : OK, Pid : 0.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Info ]-EasyClean-V1.1-Executor.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Init SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-InitInArgvSt : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Split Str To Queue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 10.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-MyStrToLong : OK, Str : 5258, Base : 10, RetVal : 5258.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-MyStrToLong : OK, Str : 2, Base : 10, RetVal : 2.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-MyStrToLong : OK, Str : 0, Base : 10, RetVal : 0.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Check Input Arg : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Connect Info : OK , DbHost : '192.168.142.12', DbUser : 'czg', DbName : 'gbase', DbPort : 5258.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Init SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-InitInArgvSt : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Split Str To Queue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-GetSqQueueLen : OK, SqQueueLen : 10.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-MyStrToLong : OK, Str : 5258, Base : 10, RetVal : 5258.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-MyStrToLong : OK, Str : 2, Base : 10, RetVal : 2.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Read SqQueue : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-MyStrToLong : OK, Str : 0, Base : 10, RetVal : 0.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Check Input Arg : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Connect Info : OK , DbHost : '192.168.142.12', DbUser : 'czg', DbName : 'gbase', DbPort : 5258.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-client character : utf8
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Sql Query : SET autocommit = 1
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-client character : utf8
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Sql Query : SET autocommit = 1
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-8a Set Parameters : OK, ParameterName : 'autocommit', ParameterVal : '1'.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Sql Query : SET gcluster_lock_timeout = 5
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-8a Set Parameters : OK, ParameterName : 'autocommit', ParameterVal : '1'.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Sql Query : SET gcluster_lock_timeout = 5
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-8a Set Parameters : OK, ParameterName : 'gcluster_lock_timeout', ParameterVal : '5'.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-8a Set Parameters : OK, ParameterName : 'gcluster_lock_timeout', ParameterVal : '5'.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Get Os Env : OK, OsEnvName : CLEAN_VOID_RATE_TOOL_HOME, RetVal : /opt/Developer/CleanVoidRate/.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Get Os Env : OK, OsEnvName : CLEAN_VOID_RATE_TOOL_HOME, RetVal : /opt/Developer/CleanVoidRate/.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-File2Key : OK, RetKey : 51f5.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-File2Key : OK, RetKey : 51f5.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-MessageGet : OK.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-MessageGet : OK.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Printf InArgvSt :
SourceHost : 192.168.142.12
SourceUser : czg
SourcePwd : qwer1234
SourceDbName : gbase
SourcePort : 5258
SourceCharset : utf8
TargetDb : bd_db_apblc
VoidRate : 0
CleanTabNum : 0
ToolPath : /opt/Developer/CleanVoidRate/
ExecutorPath :
PublicStr : /opt/Developer/CleanVoidRate/Exec/File2Key.Data
ChdProcessNum : 2
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Printf InArgvSt :
SourceHost : 192.168.142.12
SourceUser : czg
SourcePwd : qwer1234
SourceDbName : gbase
SourcePort : 5258
SourceCharset : utf8
TargetDb : bd_db_apblc
VoidRate : 0
CleanTabNum : 0
ToolPath : /opt/Developer/CleanVoidRate/
ExecutorPath :
PublicStr : /opt/Developer/CleanVoidRate/Exec/File2Key.Data
ChdProcessNum : 2
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-SqQueue Data :
Data : [ '192.168.142.12' ,'czg' ,'qwer1234' ,'gbase' ,'5258' ,'utf8' ,'2' ,'bd_db_apblc' ,'0' ,'0' ]
FrontIndex : 0
RearIndex : 10
SqQueueLen : 10
SqQueueMaxLen : 1024
Flag : STRING_TYPE_FLAG
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-SqQueue Data :
Data : [ '192.168.142.12' ,'czg' ,'qwer1234' ,'gbase' ,'5258' ,'utf8' ,'2' ,'bd_db_apblc' ,'0' ,'0' ]
FrontIndex : 0
RearIndex : 10
SqQueueLen : 10
SqQueueMaxLen : 1024
Flag : STRING_TYPE_FLAG
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-MessageRcv : OK, MsgSize : 24, MsgType : 1, MsgInfo : 'gbase8a_i2b_config_table'.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-MessageRcv : OK, MsgSize : 29, MsgType : 1, MsgInfo : 'sg_t_loadconfig_incr_20230801'.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Sql Query : LOCK TABLE bd_db_apblc.gbase8a_i2b_config_table WRITE
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Sql Query : LOCK TABLE bd_db_apblc.sg_t_loadconfig_incr_20230801 WRITE
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:20-P[43997]-T[140510946023232]-[Debug]-Sql Query : CREATE TABLE bd_db_apblc.sg_t_loadconfig_incr_20230801_COPY_TAB LIKE bd_db_apblc.sg_t_loadconfig_inc
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:20-P[43996]-T[140175809361728]-[Debug]-Sql Query : CREATE TABLE bd_db_apblc.gbase8a_i2b_config_table_COPY_TAB LIKE bd_db_apblc.gbase8a_i2b_config_table
2024-01-03 09:49:21-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 1 s, Affect 0 Rows.
2024-01-03 09:49:21-P[43997]-T[140510946023232]-[Debug]-Sql Query : INSERT INTO bd_db_apblc.sg_t_loadconfig_incr_20230801_COPY_TAB SELECT * FROM bd_db_apblc.sg_t_loadc
2024-01-03 09:49:21-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 1 s, Affect 0 Rows.
2024-01-03 09:49:21-P[43996]-T[140175809361728]-[Debug]-Sql Query : INSERT INTO bd_db_apblc.gbase8a_i2b_config_table_COPY_TAB SELECT * FROM bd_db_apblc.gbase8a_i2b_con
2024-01-03 09:49:23-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 2 s, Affect 6326 Rows.
2024-01-03 09:49:23-P[43996]-T[140175809361728]-[Debug]-Sql Query : TRUNCATE TABLE bd_db_apblc.gbase8a_i2b_config_table
2024-01-03 09:49:23-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 2 s, Affect 5325 Rows.
2024-01-03 09:49:23-P[43997]-T[140510946023232]-[Debug]-Sql Query : TRUNCATE TABLE bd_db_apblc.sg_t_loadconfig_incr_20230801
2024-01-03 09:49:23-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 6326 Rows.
2024-01-03 09:49:23-P[43996]-T[140175809361728]-[Debug]-Sql Query : INSERT INTO bd_db_apblc.gbase8a_i2b_config_table SELECT * FROM bd_db_apblc.gbase8a_i2b_config_table
2024-01-03 09:49:23-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 5325 Rows.
2024-01-03 09:49:23-P[43997]-T[140510946023232]-[Debug]-Sql Query : INSERT INTO bd_db_apblc.sg_t_loadconfig_incr_20230801 SELECT * FROM bd_db_apblc.sg_t_loadconfig_inc
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 1 s, Affect 5325 Rows.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Sql Query : UNLOCK TABLES
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Sql Query : DROP TABLE IF EXISTS bd_db_apblc.sg_t_loadconfig_incr_20230801_COPY_TAB
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 1 s, Affect 6326 Rows.
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-Sql Query : UNLOCK TABLES
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-Sql Query : DROP TABLE IF EXISTS bd_db_apblc.gbase8a_i2b_config_table_COPY_TAB
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Info ]-CleanTabVoidRate : OK, DbName : bd_db_apblc, TabName : sg_t_loadconfig_incr_20230801.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-MessageRcv : OK, MsgSize : 24, MsgType : 1, MsgInfo : 'sg_t_loadconfig_incr_odm'.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Sql Query : LOCK TABLE bd_db_apblc.sg_t_loadconfig_incr_odm WRITE
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Sql Query : CREATE TABLE bd_db_apblc.sg_t_loadconfig_incr_odm_COPY_TAB LIKE bd_db_apblc.sg_t_loadconfig_incr_odm
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Info ]-CleanTabVoidRate : OK, DbName : bd_db_apblc, TabName : gbase8a_i2b_config_table.
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-MessageRcv : OK, MsgSize : 6, MsgType : 1, MsgInfo : 'Finish'.
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-Destroy SqQueue : OK.
2024-01-03 09:49:24-P[43996]-T[140175809361728]-[Debug]-DestoryInArgvSt : OK.
2024-01-03 09:49:24-P[43992]-T[140539952031552]-[Debug]-ChkProcExitStatus : OK, Normal Termination, Pid : 43996, Exit Status : 0.
2024-01-03 09:49:24-P[43992]-T[140539952031552]-[Debug]-MyWaitPid : OK, RetPid : 43996.
2024-01-03 09:49:24-P[43992]-T[140539952031552]-[Debug]-Leave SqQueue : OK.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Sql Query : INSERT INTO bd_db_apblc.sg_t_loadconfig_incr_odm_COPY_TAB SELECT * FROM bd_db_apblc.sg_t_loadconfig
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 2191 Rows.
2024-01-03 09:49:24-P[43997]-T[140510946023232]-[Debug]-Sql Query : TRUNCATE TABLE bd_db_apblc.sg_t_loadconfig_incr_odm
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 1 s, Affect 2191 Rows.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Sql Query : INSERT INTO bd_db_apblc.sg_t_loadconfig_incr_odm SELECT * FROM bd_db_apblc.sg_t_loadconfig_incr_odm
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 2191 Rows.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Sql Query : UNLOCK TABLES
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Sql Query : DROP TABLE IF EXISTS bd_db_apblc.sg_t_loadconfig_incr_odm_COPY_TAB
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Execute Sql : OK, Sql Elapsed Time 0 s, Affect 0 Rows.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Info ]-CleanTabVoidRate : OK, DbName : bd_db_apblc, TabName : sg_t_loadconfig_incr_odm.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-MessageRcv : OK, MsgSize : 6, MsgType : 1, MsgInfo : 'Finish'.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-Destroy SqQueue : OK.
2024-01-03 09:49:25-P[43997]-T[140510946023232]-[Debug]-DestoryInArgvSt : OK.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-ChkProcExitStatus : OK, Normal Termination, Pid : 43997, Exit Status : 0.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-MyWaitPid : OK, RetPid : 43997.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-MessageCtl : OK.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-Destroy DqlResult : OK.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-Destroy SqQueue : OK.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-DestoryInArgvSt : OK.
2024-01-03 09:49:25-P[43992]-T[140539952031552]-[Debug]-Destroy SqQueue : OK.