一大早起来,发现个人网站又报警了,前两天刚加了2G内存,我了个去。。。。
上机器查日志 /var/log/message
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7539] 0 7539 43237 385 0 0 0 curl
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7540] 0 7540 19602 462 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7541] 0 7541 19601 460 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7542] 0 7542 19602 462 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7543] 0 7543 19603 463 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7544] 0 7544 19601 461 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7545] 0 7545 19602 461 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7546] 0 7546 19602 462 1 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7547] 0 7547 19602 462 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7548] 0 7548 19602 461 0 0 0 sendmail
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7549] 1000 7549 33286 2565 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7550] 1000 7550 33218 2521 1 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7551] 1000 7551 33204 2483 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7552] 1000 7552 33408 2698 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7553] 1000 7553 32843 2071 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7554] 1000 7554 33204 2483 1 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7555] 1000 7555 33205 2454 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7556] 1000 7556 32705 1833 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7557] 1000 7557 32705 1830 1 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7558] 1000 7558 32705 1834 1 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7559] 0 7559 35056 322 0 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7560] 0 7560 35056 322 1 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7561] 0 7561 35056 322 1 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7562] 0 7562 35056 322 1 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7563] 0 7563 35056 322 0 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7564] 0 7564 35056 322 1 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7565] 0 7565 35056 322 1 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7566] 0 7566 35056 322 0 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7567] 0 7567 35056 322 0 0 0 crond
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7568] 1000 7568 32705 1833 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7569] 1000 7569 32704 1641 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7570] 1000 7570 32704 1570 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7571] 1000 7571 32704 1570 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7572] 1000 7572 32704 1545 0 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: [ 7573] 0 7573 32704 1384 1 0 0 httpd
Aug 29 07:31:06 AM9YCYMHJM kernel: Out of memory: Kill process 6525 (mysqld) score 143 or sacrifice child
Aug 29 07:31:06 AM9YCYMHJM kernel: Killed process 6525, UID 27, (mysqld) total-vm:1736764kB, anon-rss:559072kB, file-rss:2044kB
看到内存被打满了,mysql进程又躺枪了(内存耗尽时,内核会选出一个消耗内存最多进程K掉,由于mysql体量大,总是被连累), 看进程信息,发现有很多crond和sendmail
代码里并没有发邮件的东西,忽然想起来昨天晚上加了七八个crontab,且执行频率较高
网上一查,发现确实是这货惹的事,原来crontab执行一次,就给所属者发邮件,邮件发给谁可配置
解决方案:
crontab -e
第一行写入MAILTO=""
表示不发邮件
保存退出
杀掉所有sendmail进程 ps -ef | grep sendmail | grep -v grep | awk '{print $2}' | xargs kill
搞定