问题描述:

一个文件正在被进程写 我想查看哪个进程在写这个文件。

 

解决思路:

Linux 下每个文件都会在某个块设备上存放,当然也都有相应的 inode ,那么透过 vfs.write 我们就可以知道谁在不停的写入特定的设备上的 inode。

那用什么工具呢? 这个神器就是systemtap ,具体的命令就是stap。 

参考:http://cccgw.info/2012/07/systemtap%E5%85%A5%E9%97%A8/

安装很简单:yum -y install systemtap

但是这个安装完毕,还需要内核的支持,不然会报错。具体的说就是需要安装kernel-debuginfo包。

安装过程:

1 查看内核版本

   uname -r

   2.6.32-279.el6.x86_64

2 下载相应的包:

   wget http://debuginfo.centos.org/6/x86_64/kernel-debuginfo-2.6.32-279.el6.x86_64.rpm

   wget http://debuginfo.centos.org/6/x86_64/kernel-debuginfo-common-2.6.32-279.el6.x86_64.rpm

3 安装rpm包:

  rpm -ivh kernel-debuginfo-2.6.32-279.el6.x86_64.rpm kernel-debuginfo-common-2.6.32-279.el6.x86_64.rpm

4 确保安装包正常:

  rpm -qa|grep kernel-debug

  kernel-debuginfo-common-x86_64-2.6.32-279.el6.x86_64

  kernel-debuginfo-2.6.32-279.el6.x86_64

 

好了,测试下stap的可用性吧:

执行命令:stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'

显示下面结果:

Pass 1: parsed user script and 83 library script(s) using 194448virt/23176res/3056shr kb, in 130usr/10sys/142real ms.

Pass 2: analyzed script: 1 probe(s), 1 function(s), 3 embed(s), 0 global(s) using 427088virt/122744res/8316shr kb, in 1400usr/90sys/1493real ms.

Pass 3: using cached /root/.systemtap/cache/4a/stap_4a1eb85edba807357c24d4e1a07bc9d7_1471.c

Pass 4: using cached /root/.systemtap/cache/4a/stap_4a1eb85edba807357c24d4e1a07bc9d7_1471.ko

Pass 5: starting run.

read performed

Pass 5: run completed in 0usr/30sys/346real ms.

 

说明stap可以用了。

用法:

stap stap预制脚本.stp major minor 文件inode 

(systemstap,major,minor ,这些不明白的去google吧)

比如我想知道我的/root/debug.log 是谁写的,例子:

 df -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/mapper/VolGroup-lv_root

                       34G   11G   21G  35% /

 

ls -l /dev/mapper/VolGroup-lv_root

lrwxrwxrwx. 1 root root 7 Aug 31  2012 /dev/mapper/VolGroup-lv_root -> ../dm-0 

 

继续找:

 

ls -l /dev/dm-0       

brw-rw----. 1 root disk 253, 0 Aug 31  2012 /dev/dm-0

 

看到了么?  253是major number ,0 是minor number

 

再找文件的inode:

 stat -c '%i' /root/debug.log

523308

 

组合命令来吧:

stap /usr/share/doc/systemtap-client-1.7/examples/io/inodewatch.stp 253 0 523302

 

astroc(740) vfs_write 0x800011/25337884

astroc(740) vfs_write 0x800011/25337884

astroc(740) vfs_write 0x800011/25337884

astroc(740) vfs_write 0x800011/25337884

------------------------------------------------------------------------------

查看进程写哪个文件

[root]# ls -l /proc/29390/fd

[root]# crash /usr/lib/debug/lib/modules/2.6.32-431.el6.x86_64/vmlinux -S /boot/System.map-2.6.32-431.el6.x86_64 

crash: cannot open /boot/System.map


crash 6.1.0-5.el6

Copyright (C) 2002-2012  Red Hat, Inc.

Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation

Copyright (C) 1999-2006  Hewlett-Packard Co

Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited

Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.

Copyright (C) 2005, 2011  NEC Corporation

Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.

Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.

This program is free software, covered by the GNU General Public License,

and you are welcome to change it and/or distribute copies of it under

certain conditions.  Enter "help copying" to see the conditions.

This program has absolutely no warranty.  Enter "help warranty" for details.

 

GNU gdb (GDB) 7.3.1

Copyright (C) 2011 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-unknown-linux-gnu"...


  SYSTEM MAP: /boot/System.map-2.6.32-431.el6.x86_64                   

DEBUG KERNEL: /usr/lib/debug/lib/modules/2.6.32-431.el6.x86_64/vmlinux (2.6.32-431.el6.x86_64)

    DUMPFILE: /dev/crash

        CPUS: 32

        DATE: Sun Jun 14 23:30:39 2015

      UPTIME: 1 days, 19:00:27

LOAD AVERAGE: 10.78, 9.96, 9.42

       TASKS: 734

    NODENAME: localhost

     RELEASE: 2.6.32-431.el6.x86_64

     VERSION: #1 SMP Fri Nov 22 03:15:09 UTC 2013

     MACHINE: x86_64  (2600 Mhz)

      MEMORY: 32 GB

         PID: 44892

     COMMAND: "crash"

        TASK: ffff88083724d500  [THREAD_INFO: ffff880838cd0000]

         CPU: 1

       STATE: TASK_RUNNING (ACTIVE)


crash> set 29399

    PID: 29399

COMMAND: "postgres"

   TASK: ffff880435ed4080  [THREAD_INFO: ffff88043747e000]

    CPU: 2

  STATE: TASK_UNINTERRUPTIBLE 

crash> files

PID: 29399  TASK: ffff880435ed4080  CPU: 0   COMMAND: "postgres"

ROOT: /    CWD: /p3/pgsql/data1

 FD       FILE            DENTRY           INODE       TYPE PATH

  0 ffff880831e78a80 ffff880439dede40 ffff880839860d48 CHR  /dev/null

  1 ffff88082ec4b080 ffff88043475c140 ffff880439f3e4e8 CHR  /dev/pts/4

  2 ffff88082ec4b080 ffff88043475c140 ffff880439f3e4e8 CHR  /dev/pts/4

  3 ffff8808389d0540 ffff880836647c00 ffff88083a28f7f8 FIFO 

  4 ffff88082fadbcc0 ffff880836647c00 ffff88083a28f7f8 FIFO 

  5 ffff880011b60600 ffff8802139aa440 ffff88041d2ba8b0 REG  /p3/pgsql/data1/pg_xlog/000000010000038A000000C7

  6 ffff88042309b440 ffff8803f479ccc0 ffff880426451bd8 FIFO 

  7 ffff88008772b180 ffff8800bc83c600 ffff8800879628b0 REG  /p3/pgsql/data1/base/16384/16408

  8 ffff8800113d4440 ffff8803f479c180 ffff8804345c94c8 SOCK 

  9 ffff880437604600 ffff880410698b40 ffff880076b4f0c0 REG  /p3/pgsql/data1/base/16384/16408.1