mysql can't init tc log_[运维]-Can't init tc log

收到邮件短信说开发环境服务器内存超过90%，打开一看，jenkins占用了178Gb的日志空间，直接将磁盘空间占满，mysql无法再写入数据，mysql挂了，jenkins挂了。找到jenkins日志目录，清空了历史Log后，重启jenkins，一切ok，重启mysql，无启动启动，报错：

86011 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] /usr/sbin/mysqld (mysqld 10.1.18-MariaDB) starting as process 4290 ...

86012 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Using mutexes to ref count buffer pool pages

86013 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: The InnoDB memory heap is disabled

86014 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

86015 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier

86016 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Compressed tables use zlib 1.2.7

86017 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Using Linux native AIO

86018 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Using SSE crc32 instructions

86019 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Initializing buffer pool, size = 128.0M

86020 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Completed initialization of buffer pool

86021 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Highest supported file format is Barracuda.

86022 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: 128 rollback segment(s) are active.

86023 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Waiting for purge to start

86024 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.32-78.1 started; log sequence number 241252941

86025 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] Plugin 'FEEDBACK' is disabled.

86026 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] Recovering after a crash using tc.log

86027 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [ERROR] Can't init tc log

86028 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [ERROR] Aborting

86029 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE

86030 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: Failed to start MariaDB database server.

86031 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: Unit mariadb.service entered failed state.

86032 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: mariadb.service failed.

去mysql目录下，查看了下:

-rw-rw---- 1 mysql mysql 16384 Apr 12 18:26 aria_log.00000001

-rw-rw---- 1 mysql mysql 52 Apr 12 18:26 aria_log_control

-rw-rw---- 1 mysql mysql 35651584 Apr 24 10:29 ibdata1

-rw-rw---- 1 mysql mysql 50331648 Apr 24 10:29 ib_logfile0

-rw-rw---- 1 mysql mysql 50331648 Apr 7 14:43 ib_logfile1

-rw-r----- 1 mysql mysql 0 Oct 27 18:38 iZ25ro2rttbZ.err

-rw-rw---- 1 mysql mysql 4 Mar 9 15:19 iZ25ro2rttbZ.pid

-rw-rw---- 1 mysql mysql 4 Apr 12 18:26 iZ2ze43t8c42mpy1sr3pfjZ.pid

-rw-rw---- 1 mysql mysql 4 Mar 13 10:11 iZ2ze7cbcapiyem9l3lzxrZ.pid

drwx------ 2 mysql mysql 4096 Apr 23 16:47 medchannel

-rw-rw---- 1 mysql mysql 0 Oct 27 18:38 multi-master.info

drwx------ 2 mysql mysql 4096 Oct 27 18:38 mysql

srwxrwxrwx 1 mysql mysql 0 Apr 12 18:26 mysql.sock

-rw-r--r-- 1 root root 15 Oct 27 18:38 mysql_upgrade_info

drwx------ 2 mysql mysql 4096 Apr 22 14:03 openmore

drwx------ 2 mysql mysql 4096 Oct 27 18:38 performance_schema

drwx------ 2 mysql mysql 4096 Nov 9 14:22 rap_db

drwx------ 2 mysql mysql 4096 Mar 9 14:57 redmine_db

-rw-rw---- 1 mysql mysql 24576 Apr 12 18:26 tc.log

drwx------ 2 mysql mysql 12288 Apr 1 16:02 wenxiaoyou_db

执行如下操作，重启一切OK：

rm -f ib_logfile*

rm -f tc.log have fun!