排查 Linux 问题

1. 排查 Linux 问题

1.1. 网络

1.1.1. 端口占用

Linux:

sudo lsof -i -P -n | grep LISTEN
sudo netstat -tulpn | grep LISTEN
sudo ss -tulpn | grep LISTEN
sudo lsof -i:22 ## see a specific port such as 22 ##
sudo nmap -sTU -O IP-address-Here

FreeBSD/macOS (OS X) netstat syntax:

netstat -anp tcp | grep LISTEN
netstat -anp udp | grep LISTEN

udo sockstat -4 -6 -l

OpenBSD netstat syntax:

netstat -na -f inet | grep LISTEN
netstat -nat | grep LISTEN

Windows:

netstat -bano | more
netstat -bano | grep LISTENING
netstat -bano | findstr /R /C:"[LISTEING]"

• 用脚本方式

(echo >/dev/tcp/localhost/23) &>/dev/null && echo "open" || echo "close"
(echo >/dev/tcp/192.168.2.20/22) &>/dev/null && echo "open" || echo "close"

#!/bin/bash
dest_box="aws-prod-server-42"
echo "Testing the ssh connectivity ... "
if ! (echo >/dev/tcp/$dest_box/22) &>/dev/null
then
    echo "$0 cannot connect to the $dest_box. Check your vpn connectivity."
else
    echo "Running the ansible playboook ..."
    ansible-playbook -i hosts --ask-vault-pass --extra-vars '@cluster.data.yml' main.yaml
fi

sh:

nc -w {timeout} -zv {server_IP_hostname} {tcp_port} &>/dev/null && echo "Open" || echo "Close"
nc -w 5 -zv 192.168.2.20 23 &>/dev/null && echo "TCP/23 Open" || echo "TCP/23 Close"

#!/bin/bash
dest_box="aws-prod-server-42"
timeout="5" # timeouts in seconds
echo "Testing the ssh connectivity in $timeout seconds ... "
# make sure 'nc' is installed, else die ..
if ! type -a nc &>/dev/null
then
    echo "$0 - nc command not found. Please install nc and run the script again."
    exit 1
fi
if !  nc -w "$timeout" -zv "${dest_box}" 22  &>/dev/null
then
    echo "$0 cannot connect to the $dest_box. Check your vpn connectivity."
    exit 1
else
    echo "Running the ansible playboook ..."
    ansible-playbook -i hosts --ask-vault-pass --extra-vars '@cluster.data.yml' main.yaml
fi

perl:

#!/usr/bin/perl -w 
use IO::Socket::INET;
 
# Set server name and port here
$my_server="192.168.2.20";
$my_server_tcp_port="22";
 
# make a new object
my $server_test = IO::Socket::INET->new(
  PeerAddr => "$my_server",
  PeerPort => "$my_server_tcp_port",
  Proto => 'tcp',
  Timeout => 5
);
 
# test it and die or continue as per your needs
if ($server_test) {
  print "TCP port $my_server_tcp_port is open for the $my_server.\n";
  print "Now doing something ...\n";
  close $server_test;
} 
else {
  print "TCP port $my_server_tcp_port is closed or timed out for the $my_server.\n";
}

python:

#!/usr/bin/python3
# Tested on Python 3.6.xx and 3.8.xx only (updated from Python 2.x)
import socket
 
# Create a new function 
def check_server_tcp_port(my_host_ip_name, my_tcp_port, timeout=5):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(timeout)
    try:
        s.connect((my_host_ip_name, my_tcp_port))
        print(f"TCP port {my_tcp_port} is open for the {my_host_ip_name}.")
        s.close()
        return True
    except socket.timeout:
        print(f"TCP port {my_tcp_port} is closed or timed out for the {my_host_ip_name}.")
        return False
 
# Test it 
check_server_tcp_port("localhost", 22)
check_server_tcp_port("192.168.2.20", 22)

• man

man lsof
man ss
man netstat
man nmap
man 5 services
man nc

参考自 这里

1.2. 内存

# 查前 3 占用最高
ps -o pid,user,%mem,command ax | sort -b -k3 -r

1.3. 硬盘

# 查看总体大小
df -h

# 查看目录下每个文件夹的大小
du -h --max-depth=1

# 查看子目录文件及文件夹大小统计值
du -sh

1.4. 文件

1.4.1. Linux 之查看文件被哪个进程占用 fuser/lsof/pidof

• fuser

<0>. 查看某个进程的 pid
# fuser /usr/bin/pulseaudio 
/usr/bin/pulseaudio: 11206e

<1>. 查看当前目录正在被哪些进程在使用
# fuser -uv .
用户     进程号 权限   命令
/home/test:       test    3022 ..c.. (test)gnome-session-b
                  test    3053 ..c.. (test)dbus-daemon

<2>. 查看/lib/x86_64-linux-gnu/libc-2.27.so 正在被哪些进程在使用
# fuser -uv /lib/x86_64-linux-gnu/libc-2.27.so
用户     进程号 权限   命令
/lib/x86_64-linux-gnu/libc-2.27.so:
                     test    2983 ....m (test)systemd
                     test    3022 ....m (test)gnome-session-b
                     test    3047 ....m (test)fcitx
                     test    3053 ....m (test)dbus-daemon
                     test    3073 ....m (test)dbus-daemon

<3>. 查看/proc 这个目录有哪些进程在使用
# fuser -uv /proc
                     用户     进程号 权限   命令
/proc:               root     kernel mount (root)/proc
                     test    8736 f.... (test)nacl_helper
                     test    8739 f.... (test)chrome
                     test    8789 f.... (test)chrome
                     test   10621 f.... (test)Typora
                     test   10654 f.... (test)Typora

<4>. 那些进程在进行/proc 文件系统的读取
# fuser -muv /proc
                     用户     进程号 权限   命令
/proc:               root     kernel mount (root)/proc
                     test    2983 f.... (test)systemd
                     test    3258 f.... (test)gvfs-udisks2-vo
                     test    3369 f.... (test)gsd-housekeepin
                     test    4185 f.... (test)gnome-software
                     test    4754 f.... (test)gvfsd-trash
                     test    8735 .rc.. (test)chrome
                     test    8736 frc.. (test)nacl_helper

<5>. 杀死/home 占用 home 目录的所有进程
# fuser -mki /home

• lsof

1. 查看那些进程占用/lib/x86_64-linux-gnu/libc-2.27.so
# lsof |grep /lib/x86_64-linux-gnu/libc-2.27.so

• pidof

1. 查看某个进程的 pid
# pidof pulseaudio 
11206