Linux System Admin Handbook - Settings & Monitoring

Linux System Admin - Settings & Monitoring

---

[ Kernel Version]

uname -a (or /proc/version) : kernel version and release
/proc/cmdline : kernel parameters (by bootloader)

[ Release Version ]

/etc/redhat-release or centos-release or system-release
/etc/lsb-release (for ubuntu)
/etc/openwrt-release (for openwrt)

[ Source Code ]

/usr/src/kernels : kernel source code
/usr/include : header files for kernel and normal software
/usr/local/include : header files for local user software

[ Libraries ]

/lib : libs for the kernel and basic os (/bin, /sbin)
/lib/modules : kernel modules

/usr/lib : libs for the normal software (/usr/bin, /usr/sbin)
/usr/local/lib : libs for the local user sofware (/usr/local/bin, /usr/local/sbin)

---

ldconfig -p : display the .so file list (used by the os in loading executables).
ldconfig (-v) : update the .so file list, according to /etc/ld.so.conf.
LD_LIBRARY_PATH : environment variable, additional .so file paths. OBSOLETE!

ldd <file-name> : to display .so files used by a executable.

[ Hardware Information ]

lscpu or /proc/cpuinfo : display the cpu information.
free -m : display the memory information.
lspci -v : display all the info about connected pci devices.
lsusb [-v] : display all the info about connected USB devices.
lsblk : display all the block devices.
dmidecode : display BIOS related info.

[optional] lshw -short / hwinfo -short : show HW info in man-readable format.

[ Kernel Modules and Drivers ]

lsmod : list all the loaded kernel modules.
/lib/modules/$(uname -r) : kernel modules and settings.
modinfo <module-name> : display info about a kernel module

modprobe <module-name> [parameters] : to load a kernel module (/etc/modprobe.d/ for config files).
modprobe -r <module-name> : to unload a kernel module.

depmod : to generate module dependant list.

insmod <module-name> rmmod <module-name> : to load/unload a kernel module, without considering dependence.

/sys : sysfs, a map of (a) loaded kernel module (b) device tree (c) kernel.

udevd : dynamic device management (in user space). it listens to the uevent (from kernel), read device information from /sys, and create/update/remove device files in /dev according to user defined rules.
/etc/udev : files to control the create/update of device files (in /dev/)

[ Performance Monitoring ]

vmstat -S M [interval] [count] : display the overall system status (memory, cpu load, io load, swap).

free -m : display the memory usage.
df -h : display free space (file system).

top : real time performance monitor.

iostat -mtxz -p : I/O statistics. pay attention to %util, await, svctm and avgqu-sz.
ifconfig : pay attention to counters (for package drop/error/overrun/collisions).

[ Storage ]

lsbk : list all the partitions on block devices.
blkid : list partitions, uuid and filesystem type.
/proc/mtd : list all the partitions in the on-board flash.
fdisk <dev> : to manage partitions and fs on a block device.

/sbin/hdparm : get/set SATA/IDE device parameters (refer to man for details).

[ File-System ]

mount : list all the mounted file system.
stat -f <path> : display information about a file system.
fsck (e2fsck) [dev] : check and repair a file system.

df -h : list free space (in each mounted file system).
du -hs <dir> : display the size of a directory.

sync : flush buffered info to disk.
echo 3 > /proc/sys/vm/drop_caches : free caches.

[ File Management ]

stat <file-name> : display information about a file.

dd if=<file> of=<file> bs=<block_size, bytes> count=<number of blocks : copy raw data
tar : backup / restore
od -t x1 -c <file-name> : dump file in hex format

[ init ]

/sbin/init : upstart process mngt daemon.
/etc/inittab : config file.
/etc/rc.d : service scripts for init.

chkconfig: list / set upstart services.

[ Log ]

/var/log : all the logs

/var/log/dmesg : messages from kernel ring buffer.
/var/log/messages* : logs by syslogd (rsyslogd)

/var/log/utmp, wtmp, btmp : logs for current login, all the login/logout, failed login.
/var/log/lastlog : each user’s last login info.
/var/log/secure* : logs by sshd.

dmesg : messages from kernel ring buffer.

w : display who is currently logged and what they are doing (/var/log/utmp).
last : display all the login/logout/boot logs (/var/log/wtmp,btmp).
lastlog : display each user’s last login info (/var/log/lastlog).

/sbin/rsyslogd, /sbin/syslogd : the proc in charge of logging.
/usr/bin/logger : program to write logs.

[ Crontab ]

/usr/bin/crond : crond service
/etc/cron* : crond related config files

crontab -l : list all the active crontab entries.
crontab -e : edit crontab entries.

[ Process Management ]

ps -elf[F] : list all processes with detailed info, (F: thread view).
/proc : system and process related information.

kill -9 <pid> : kill a process.

time -v <command> : run a program and summarize the system resource usage.
strace <command> / strace -p <pid> : log system calls made by a process.

lsof <filename> : list processes that open the file.
lsof +d/+D <dir> : list processes that open files in a dir.
lsof -p <pid> : list files opened by a process.
lsof -c <command> : list files opened by a process.

[ IPC ]

ipcs, ipcs -p : display info about active IPC objects (shared memory, semaphore, and message queue).
ipcrm : remove a System V IPC object (shared memory, semaphore, and message queue).

[ RPM & Yum ]

rpm -qa : list all the installed rpm packages.

rpm -qi <package-name> : display info about a installed rpm package.
rpm -ev <package_name> : to remove a package from the system.

rpm -qf <file-name> : to find the installed rpm package that include a file.

rpm -qpl <package-file> : list files included in a rpm package file.
rpm -ivh / Uvh <package-file> : install/upgrade a rpm package.
rpm2cpio <package-file> | cpio -div : unpack the rpm package into the current directory.

yum search <keyword> : to find a rpm package
yum install <package-name> : to install a rpm package.
yum update / upgrade: update installed software.

[ Virtual Memory Management ]

/sbin/mkswap : set up swap area.
/sbin/swapon, swapoff : enable/disable swap.

[ Network Management ]

ifconfig -a : display information about all the network interfaces.
route -n : display routing table.
arp -n : display the ARP table.
nslookup <domain-name> : get IP address from domain name.

ifconfig <if-name> up/down : up/down an interface (like eth0:1)
ifconfig <eth0:1> <192.168.1.143> <netmask 255.255.255.0> : add an address to eth0.

route add -net 192.56.76.0 netmask 255.255.255.0 dev eth0 : add a route
route add default gw 192.168.5.1 : add a default route.

netstat -anp : display all the socket statistics and information.
//path for unix socket: @ for abstract name.

getenforce : check if SELinux is enabled.
service iptables start/stop : start/stop iptables.
iptables -v -l : show firewall related rules
iptables -t nat -v -L : show nat rules
tc -d qdisc : to show qdisc (traffic control) related information.

conntrack -L : display all the connections tracked by netfilter.

iwconfig / iw phy, iw dev : display wireless related settings.

[ User Management ]

useradd usermod userdel : add/modify/delete a user.
groupadd groupmod groupdel : add/modify/delete a group.

/etc/passwd : user list
/etc/group : group list
/etc/gshadow : group password
/etc/shadow : user password

sudo <command> : run a command as root
sudo -l : list the commands a user can run as root
/etc/sudoers : config file for sudo.

[ Shell ]

env : list all the environment variables.
set : list all the (local and environment) variables.