最近在灰度测试PHP7的过程中,php-fpm出现间歇性的段错误。系统的错误信息如下:
php-fpm[7664]: segfault at 7f6ff4600000 ip 00007f6ff782176f sp 00007fff2e9c2fe8 error 4 in libc-2.12.so[7f6ff7798000+18a000]
为了排查出错的原因,我们接下来需要进行调试。由于错误间歇性出现在php-fpm处理请求的过程中,因此,我们需要获取获取Linux的core dumps文件。
打开Linux的core dumps
一般情况下,Linux默认core dumps是关闭状态。我们可以将其打开并且重定向到我们指定的文件。
$echo '/tmp/coredump-%e.%p' > /proc/sys/kernel/core_pattern
core dumps文件支持变量:
%%a single % character
%c core file size soft resource limit of crashing process (since
Linux 2.6.24)%d dump mode—same as value returned by prctl(2) PR_GET_DUMPABLE (since Linux 3.7)%e executable filename (without path prefix)%E pathname of executable, with slashes ('/') replaced by
exclamation marks ('!') (since Linux 3.0).
%g (numeric) real GID of dumped process
%h hostname (same as nodename returned by uname(2))%p PID of dumped process, as seen inthe PID namespace inwhich
the process resides
%P PID of dumped process, as seen inthe initial PID namespace (since Linux 3.12)%s number of signal causing dump
%t timeof dump, expressed as seconds since the Epoch,
1970-01-01 00:00:00 +0000 (UTC)%u (numeric) real UID of dumped process
这个例子中,我们把错误文件重定向到/tmp目录下。
配置php-fpm支持core dumps
为了让php-fpm支持core dumps,我们需要打开php-fpm连接池的rlimit_core配置,在配置文件中设置。
rlimit_core = unlimited
重启php-fpm进程,当SIGSEGV信号量产生时,将会在你指定的core dumps目录产生指定的文件:
$ls /tmp/coredump*-rw------- 1 user group 220M /tmp/coredump-php-fpm.2393
使用gdb读取core dumps文件
首先,确认你的机器中正确安装了gdb调试工具(yum install gdb)。然后,你将使用gdb $program-path $coredump-path这样的命令格式调试。由于我们的程序运行在php-fpm,我们将使用以下的命令调试:
$gdb /usr/local/services/php7/sbin/php-fpm core.6054
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1)Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
...
Core was generated by `php-fpm: pool www '. Program terminated with signal 11, Segmentation fault. #0 0x00007f54017dc76f in memcpy () from /lib64/libc.so.6 ... (gdb) bt #0 0x00007f54017dc76f in memcpy () from /lib64/libc.so.6 #1 0x00007f53fdf96443 in zend_string_init (execute_data=0x7f53fe416fc0) at /usr/local/services/php7/include/php/Zend/zend_string.h:159 #2 hp_execute_ex (execute_data=0x7f53fe416fc0) at /usr/local/src/xhprof-php7/extension/xhprof.c:1476 #3 0x00000000008c28b0 in ZEND_DO_FCALL_SPEC_HANDLER () at /data/software/php-7.0.6/Zend/zend_vm_execute.h:800 #4 0x00000000008851cb in execute_ex (ex=Unhandled dwarf expression opcode 0xf3 ) at /data/software/php-7.0.6/Zend/zend_vm_execute.h:414
bt命令将会显示core dumps文件的调用栈。到此为止,我们定位到问题出现在/usr/local/src/xhprof-php7/extension/xhprof.c文件在调用memcpy()方法时,出现内存段错误。
结语
目前PHP7官方并未支持xhprof扩展,为了分析性能,我们安装了第三方编译过的版本。将该扩展从灰度环境中下线,段错误问题就不再出现了。