作者介绍:简历上没有一个精通的运维工程师。希望大家多多关注作者,下面的思维导图也是预计更新的内容和当前进度(不定时更新)。
Linux进阶部分又分了很多小的部分,我们刚讲完了Linux基础软件,下面是Linux日志。Linux 系统中的日志是记录系统活动和事件的重要工具,它们可以帮助管理员监视系统状态、调查问题以及了解系统运行状况。主要涉及到系统日志,登录日志,定时任务日志,监控日志,崩溃日志,二进制日志等内容,这些日志都存储在/var/log目录下,有的日志文本格式,可以直接使用前面学到的tail cat 等命令分析,有的日志是二进制格式需要专门的命令才能解释,比如sa journal等。我们主要从以下几个方面来介绍Linux的日志情况。
1.Linux日志-message日志(本章节)
2.Linux日志-secure日志
3.Linux日志-btmp日志
4.Linux日志-wtmp日志
5.Linux日志-lastlog日志
6.Linux日志-cron日志
7.Linux日志-sa日志
8.Linux日志-journal日志
9.Linux日志-dmesg日志
10.Linux日志-kdump日志
11.Linux日志-日志小结
在Linux系统中,message
日志通常指的是系统日志(system logs)中的一部分,这些日志记录了系统和应用程序的各种信息、警告和错误。message日志也是我们在日常运维中查看使用最最频繁的日志,没有之一。是属于文本格式的日志,可以直接用Linux操作普通文件的命令来分析。主要记录以下日志
-
系统启动和关机信息:记录系统何时启动和关机,以及相关的详细信息。
-
服务启动和停止信息:记录各种系统服务(如网络服务、数据库服务等)的启动、停止或重启的信息。
-
内核消息:包括关于硬件、驱动程序和内核的运行时消息。
-
一般的系统运行消息:例如系统运行时的一般信息、警告和调试信息。
日志基本信息
-
日志路径:/var/log/message
-
日志格式: 文本格式
-
查看方法:普通查看文件命令,tail cat vi 等命令
关机日志
#关机日志
Jul 2 15:37:08 iZ2vci40gfjzarlead7vliZ systemd-shutdown[1]: Could not kill 24231: Operation not permitted
Jul 2 15:37:08 iZ2vci40gfjzarlead7vliZ systemd: Reached target Final Step.
Jul 2 15:37:08 iZ2vci40gfjzarlead7vliZ systemd: Starting Reboot...
Jul 2 15:37:08 iZ2vci40gfjzarlead7vliZ systemd: Closed LVM2 metadata daemon socket.
Jul 2 15:37:08 iZ2vci40gfjzarlead7vliZ systemd: Shutting down.
Jul 2 15:37:08 iZ2vci40gfjzarlead7vliZ journal: Journal stopped
开机日志
#开机日志
Jul 2 23:37:11 iZ2vci40gfjzarlead7vliZ journal: Runtime journal is using 8.0M (max allowed 87.8M, trying to leave 131.7M free of 870.2M available → current limit 87.8M).
Jul 2 23:37:11 iZ2vci40gfjzarlead7vliZ kernel: Initializing cgroup subsys cpuset
Jul 2 23:37:11 iZ2vci40gfjzarlead7vliZ kernel: Initializing cgroup subsys cpu
Jul 2 23:37:11 iZ2vci40gfjzarlead7vliZ kernel: Initializing cgroup subsys cpuacct
Jul 2 23:37:11 iZ2vci40gfjzarlead7vliZ kernel: Linux version 3.10.0-1127.19.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Tue Aug 25 17:23:54 UTC 2020
服务日志
#服务的停止启动日志-docker
Jul 2 15:45:38 iZ2vci40gfjzarlead7vliZ systemd: Stopping Docker Application Container Engine...
Jul 2 15:45:38 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:38.961371914+08:00" level=info msg="Processing signal 'terminated'"
Jul 2 15:45:38 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:38.962879388+08:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
Jul 2 15:45:38 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:38.963313511+08:00" level=info msg="Daemon shutdown complete"
Jul 2 15:45:38 iZ2vci40gfjzarlead7vliZ systemd: Stopped Docker Application Container Engine.
Jul 2 15:45:38 iZ2vci40gfjzarlead7vliZ systemd: Starting Docker Application Container Engine...
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.056008354+08:00" level=info msg="Starting up"
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.104754642+08:00" level=info msg="[graphdriver] using prior storage driver: overlay2"
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.113494268+08:00" level=info msg="Loading containers: start."
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.290749025+08:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.342745667+08:00" level=info msg="Loading containers: done."
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.366203632+08:00" level=info msg="Docker daemon" commit=8e96db1 containerd-snapshotter=false storage-driver=overlay2 version=26.1.3
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.366349128+08:00" level=info msg="Daemon has completed initialization"
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:45:39.395771597+08:00" level=info msg="API listen on /run/docker.sock"
Jul 2 15:45:39 iZ2vci40gfjzarlead7vliZ systemd: Started Docker Application Container Engine.
内核日志
#内核日志
Jul 2 15:37:25 iZ2vci40gfjzarlead7vliZ kernel: TECH PREVIEW: Overlay filesystem may not be fully supported.#012Please review provided documentation for limitations.
Jul 2 15:37:26 iZ2vci40gfjzarlead7vliZ kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
Jul 2 15:37:26 iZ2vci40gfjzarlead7vliZ kernel: Bridge firewalling registered
Jul 2 15:37:26 iZ2vci40gfjzarlead7vliZ kernel: nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Jul 2 15:37:26 iZ2vci40gfjzarlead7vliZ kernel: Netfilter messages via NETLINK v0.30.
Jul 2 15:37:26 iZ2vci40gfjzarlead7vliZ kernel: ctnetlink v0.93: registering with nfnetlink.
手工命令日志
#手工修改日志
[root@iZ2vci40gfjzarlead7vliZ ~]# date -s 15:49:00
Tue Jul 2 15:49:00 CST 2024
#日志记录
cat /var/log/messages|grep "Time has"
Jul 2 15:49:00 iZ2vci40gfjzarlead7vliZ systemd: Time has been changed
分析日志
#是看不出来为什么启动失败的
[root@iZ2vci40gfjzarlead7vliZ ~]# systemctl restart docker
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
[root@iZ2vci40gfjzarlead7vliZ ~]# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Tue 2024-07-02 15:51:55 CST; 5s ago
Docs: https://docs.docker.com
Process: 1892 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
Main PID: 1892 (code=exited, status=1/FAILURE)
Jul 02 15:51:52 iZ2vci40gfjzarlead7vliZ systemd[1]: Failed to start Docker Application Container Engine.
Jul 02 15:51:52 iZ2vci40gfjzarlead7vliZ systemd[1]: Unit docker.service entered failed state.
Jul 02 15:51:52 iZ2vci40gfjzarlead7vliZ systemd[1]: docker.service failed.
Jul 02 15:51:55 iZ2vci40gfjzarlead7vliZ systemd[1]: docker.service holdoff time over, scheduling restart.
Jul 02 15:51:55 iZ2vci40gfjzarlead7vliZ systemd[1]: Stopped Docker Application Container Engine.
Jul 02 15:51:55 iZ2vci40gfjzarlead7vliZ systemd[1]: start request repeated too quickly for docker.service
Jul 02 15:51:55 iZ2vci40gfjzarlead7vliZ systemd[1]: Failed to start Docker Application Container Engine.
Jul 02 15:51:55 iZ2vci40gfjzarlead7vliZ systemd[1]: Unit docker.service entered failed state.
Jul 02 15:51:55 iZ2vci40gfjzarlead7vliZ systemd[1]: docker.service failed.
#分析message日志,日志已经已经明显指出错误在哪里。
Jul 2 15:50:43 iZ2vci40gfjzarlead7vliZ chronyd[564]: Selected source 100.100.61.88
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ systemd: Stopping Docker Application Container Engine...
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:51:48.452174007+08:00" level=info msg="Processing signal 'terminated'"
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:51:48.455067043+08:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ dockerd: time="2024-07-02T15:51:48.455271370+08:00" level=info msg="Daemon shutdown complete"
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ systemd: Stopped Docker Application Container Engine.
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ systemd: Starting Docker Application Container Engine...
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ dockerd: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives don't match any configuration option: re1gistry-mirrors
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ systemd: docker.service: main process exited, code=exited, status=1/FAILURE
Jul 2 15:51:48 iZ2vci40gfjzarlead7vliZ systemd: Failed to start Docker Application Container Engine.
总结
1.message日志是最重要的分析系统情况的日志,没有之一。
2.message日志记录的字段里面记录,时间,主机名,用户(用这个用户可以更明显的确认日志来源)
3.这个用户有kernel,systemd,journal,应用程序(docker,nginx这些)
3.分析系统崩溃,程序启动失败等都会用到他
4.甚至在根分区磁盘不够的情况下,首先要删除的也是message日志
5.参与logrotate的日志轮转进行更新,避免把磁盘打满。
关注微信公众号《运维小路》获取更多内容。