aliyun ubuntu pstack无法使用的问题解决 crawl: Input/output error

       好久没写博客了。

         最近项目组在阿里云上面开发一套分布式的智慧社区管控平台，主要是实现对楼宇对讲设备、voip终端设备、移动app的统一管控。支持用户通过手机等移动端设备来远程管理家中的终端设备，包括视频监控、sip音视频对话、设备的告警实时通知、处理等。

         以前没用过阿里云，整个开发过程还是有一些与之前在物理机上不一样的。系统主要用ubuntu 14.04.2，下面主要讲一下在aliyun主机上pstack使用时碰到的一个问题及其处理方式。。

         首先aliyun并没有安装pstack，用apt-get下载pstack的源之后，使用过程中发生错误，如下：

(No symbols found in /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2)
(No symbols found in /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2)
(No symbols found in /lib/x86_64-linux-gnu/libpng12.so.0)
(No symbols found in /usr/lib/x86_64-linux-gnu/libgnutls.so.26)
(No symbols found in /lib/x86_64-linux-gnu/libgcrypt.so.11)
(No symbols found in /usr/lib/x86_64-linux-gnu/libsasl2.so.2)
(No symbols found in /usr/lib/x86_64-linux-gnu/libgssapi.so.3)
(No symbols found in /usr/lib/x86_64-linux-gnu/libtasn1.so.6)
(No symbols found in /usr/lib/x86_64-linux-gnu/libp11-kit.so.0)
(No symbols found in /lib/x86_64-linux-gnu/libgpg-error.so.0)
(No symbols found in /usr/lib/x86_64-linux-gnu/libheimntlm.so.0)
(No symbols found in /usr/lib/x86_64-linux-gnu/libkrb5.so.26)
(No symbols found in /usr/lib/x86_64-linux-gnu/libasn1.so.8)
(No symbols found in /usr/lib/x86_64-linux-gnu/libhcrypto.so.4)
(No symbols found in /usr/lib/x86_64-linux-gnu/libroken.so.18)
(No symbols found in /usr/lib/x86_64-linux-gnu/libffi.so.6)
(No symbols found in /usr/lib/x86_64-linux-gnu/libwind.so.0)
(No symbols found in /usr/lib/x86_64-linux-gnu/libheimbase.so.1)
(No symbols found in /usr/lib/x86_64-linux-gnu/libhx509.so.5)
(No symbols found in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0)
(No symbols found in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
(No symbols found in /lib/x86_64-linux-gnu/libgcc_s.so.1)
0x7f1f050eb8b3: _fini + 0x7f1efed506eb (1d17c9000000000, 0, 1c7c13800000000, 1ca937800000000, 1ca7b8800000000, 47221000000000) + ffff80036e1f8e0c
crawl: Input/output error
Error tracing through process 4320

          排查了很久，应用进程没问题。后面查看pstack到底是什么样的。cat /usr/bin/pstack之后，发现pstack脚本时一堆乱码。我们知道pstack的工作原理其实就是一个shell脚本，然后在脚本里面调用gdb来实现对应用进程各个线程堆栈的打印，明白了这个原理之后，解决方向就很明确了。从本地一台centos物理主机上拷贝pstack的脚本覆盖掉aliyun主机上的pstack脚本即可。覆盖之后，测试一次ok。注：系统安装的pstack默认是在/usr/bin/pstack。

          附上pstack脚本的内容：

#!/bin/sh

if test $# -ne 1; then
    echo "Usage: `basename $0 .sh` <process-id>" 1>&2
    exit 1
fi

if test ! -r /proc/$1; then
    echo "Process $1 not found." 1>&2
    exit 1
fi

# GDB doesn't allow "thread apply all bt" when the process isn't
# threaded; need to peek at the process to determine if that or the
# simpler "bt" should be used.

backtrace="bt"
if test -d /proc/$1/task ; then
    # Newer kernel; has a task/ directory.
    if test `/bin/ls /proc/$1/task | /usr/bin/wc -l` -gt 1 2>/dev/null ; then
	backtrace="thread apply all bt"
    fi
elif test -f /proc/$1/maps ; then
    # Older kernel; go by it loading libpthread.
    if /bin/grep -e libpthread /proc/$1/maps > /dev/null 2>&1 ; then
	backtrace="thread apply all bt"
    fi
fi

GDB=${GDB:-/usr/bin/gdb}

if $GDB -nx --quiet --batch --readnever > /dev/null 2>&1; then
    readnever=--readnever
else
    readnever=
fi

# Run GDB, strip out unwanted noise.
$GDB --quiet $readnever -nx /proc/$1/exe $1 <<EOF 2>&1 | 
set width 0
set height 0
set pagination no
$backtrace
EOF
/bin/sed -n \
    -e 's/^\((gdb) \)*//' \
    -e '/^#/p' \
    -e '/^Thread/p'