[运维]-Can't init tc log

收到邮件短信说开发环境服务器内存超过90%，打开一看，jenkins占用了178Gb的日志空间，直接将磁盘空间占满，mysql无法再写入数据，mysql挂了，jenkins挂了。找到jenkins日志目录，清空了历史Log后，重启jenkins，一切ok，重启mysql，无启动启动，报错：

86011 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] /usr/sbin/mysqld (mysqld 10.1.18-MariaDB) starting as process 4290 ...
86012 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Using mutexes to ref count buffer pool pages
86013 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: The InnoDB memory heap is disabled
86014 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
86015 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
86016 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Compressed tables use zlib 1.2.7
86017 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Using Linux native AIO
86018 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Using SSE crc32 instructions
86019 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Initializing buffer pool, size = 128.0M
86020 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Completed initialization of buffer pool
86021 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Highest supported file format is Barracuda.
86022 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: 128 rollback segment(s) are active.
86023 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB: Waiting for purge to start
86024 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.32-78.1 started; log sequence number 241252941
86025 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] Plugin 'FEEDBACK' is disabled.                                                                                           
86026 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [Note] Recovering after a crash using tc.log
86027 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [ERROR] Can't init tc log
86028 Apr 24 10:53:55 iZ2ze7cbcapiyem9l3lzxrZ mysqld: 2017-04-24 10:53:55 140592027060352 [ERROR] Aborting
86029 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE
86030 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: Failed to start MariaDB database server.
86031 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: Unit mariadb.service entered failed state.
86032 Apr 24 10:53:57 iZ2ze7cbcapiyem9l3lzxrZ systemd: mariadb.service failed.

去mysql目录下，查看了下:

-rw-rw---- 1 mysql mysql    16384 Apr 12 18:26 aria_log.00000001
-rw-rw---- 1 mysql mysql       52 Apr 12 18:26 aria_log_control
-rw-rw---- 1 mysql mysql 35651584 Apr 24 10:29 ibdata1
-rw-rw---- 1 mysql mysql 50331648 Apr 24 10:29 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 Apr  7 14:43 ib_logfile1
-rw-r----- 1 mysql mysql        0 Oct 27 18:38 iZ25ro2rttbZ.err
-rw-rw---- 1 mysql mysql        4 Mar  9 15:19 iZ25ro2rttbZ.pid
-rw-rw---- 1 mysql mysql        4 Apr 12 18:26 iZ2ze43t8c42mpy1sr3pfjZ.pid
-rw-rw---- 1 mysql mysql        4 Mar 13 10:11 iZ2ze7cbcapiyem9l3lzxrZ.pid
drwx------ 2 mysql mysql     4096 Apr 23 16:47 medchannel
-rw-rw---- 1 mysql mysql        0 Oct 27 18:38 multi-master.info
drwx------ 2 mysql mysql     4096 Oct 27 18:38 mysql
srwxrwxrwx 1 mysql mysql        0 Apr 12 18:26 mysql.sock
-rw-r--r-- 1 root  root        15 Oct 27 18:38 mysql_upgrade_info
drwx------ 2 mysql mysql     4096 Apr 22 14:03 openmore
drwx------ 2 mysql mysql     4096 Oct 27 18:38 performance_schema
drwx------ 2 mysql mysql     4096 Nov  9 14:22 rap_db
drwx------ 2 mysql mysql     4096 Mar  9 14:57 redmine_db
-rw-rw---- 1 mysql mysql    24576 Apr 12 18:26 tc.log
drwx------ 2 mysql mysql    12288 Apr  1 16:02 wenxiaoyou_db

执行如下操作，重启一切OK：

rm -f ib_logfile*
rm -f tc.log

have fun!