更换内核后运行 qrcreator 报非法指令之调试过程
背景:
在中标5.0系统上qtcreator 运行正常,由于中标5.0 自带内核为3.10比较旧,故采用我们自己移植的4.19。结果换了内核之后qtcreator 运行报非法指令。
主要思路为通过gdb 找出产生非法指令的语句。
具体过程如下:
[sancog@localhost ~]$ qtcreator
非法指令
[sancog@localhost ~]$ whereis qtcreator
qtcreator: /usr/bin/qtcreator /usr/lib64/qtcreator /usr/libexec/qtcreator /usr/share/qtcreator
[sancog@localhost ~]$ cd /usr/bin/
[sancog@localhost bin]$ gdb ./qtcreator
GNU gdb (GDB) NeoKylin 7.8.2-39.nd7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "mips64el-neokylin-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./qtcreator...Reading symbols from /usr/bin/./qtcreator...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Missing separate debuginfos, use: debuginfo-install qt-creator-4.3.1-2.nd7.mips.4.mips64el
(gdb) r
Starting program: /usr/bin/qtcreator
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib64/libthread_db.so.1".
[New Thread 0xfff247f0a0 (LWP 5059)]
[New Thread 0xfff0de30a0 (LWP 5060)]
[New Thread 0xffe9f970a0 (LWP 5062)]
[New Thread 0xffe97970a0 (LWP 5063)]
[New Thread 0xffe8f970a0 (LWP 5064)]
[New Thread 0xffdffff0a0 (LWP 5065)]
[Thread 0xffdffff0a0 (LWP 5065) exited]
[Thread 0xffe8f970a0 (LWP 5064) exited]
[New Thread 0xffe8f970a0 (LWP 5067)]
[New Thread 0xffdffff0a0 (LWP 5068)]
Program received signal SIGILL, Illegal instruction.
0x000000fff623624c in QV4::Moth::VME::run(QV4::ExecutionEngine*, unsigned char const*, void***) () from /usr/bin/../lib64/qtcreator/../libQt5Qml.so.5
反汇编找到地址 0x000000fff623624c 的代码
(gdb) disassemble 0x000000fff623624c
Dump of assembler code for function _ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv:
0x000000fff623615c <+7140>: lwu a0,32(s4)
0x000000fff6236160 <+7144>: b 0xfff6234b28 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+1456>
0x000000fff6236164 <+7148>: lwu a1,36(s4)
0x000000fff6236168 <+7152>: lwu v0,8(s4)
0x000000fff623616c <+7156>: li a3,1
0x000000fff6236170 <+7160>: lwu a2,12(s4)
0x000000fff6236174 <+7164>: lwu a0,16(s4)
0x000000fff6236178 <+7168>: dsll v0,v0,0x3
0x000000fff623617c <+7172>: lwu a1,20(s4)
0x000000fff6236180 <+7176>: gsldx v0,0(s5,v0)
0x000000fff6236184 <+7180>: dsll a2,a2,0x3
0x000000fff6236188 <+7184>: dsll a0,a0,0x3
0x000000fff623618c <+7188>: gsldx a0,0(s5,a0)
0x000000fff6236190 <+7192>: dsll a1,a1,0x3
0x000000fff6236194 <+7196>: daddu v0,v0,a2
0x000000fff6236198 <+7200>: ld a2,0(v0)
0x000000fff623619c <+7204>: daddu a1,a0,a1
0x000000fff62361a0 <+7208>: dsrl32 a0,a2,0x11
0x000000fff62361a4 <+7212>: beq a0,a3,0xfff6236498 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+7968>
0x000000fff62361a8 <+7216>: daddiu s7,s4,32
0x000000fff62361ac <+7220>: beqz a0,0xfff6236988 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+9232>
0x000000fff62361b0 <+7224>: ld t9,8088(gp)
0x000000fff62361b4 <+7228>: ld v0,0(a1)
0x000000fff62361b8 <+7232>: dsrl32 a3,v0,0x11
0x000000fff62361bc <+7236>: dsrl32 a4,a2,0x0
0x000000fff62361c0 <+7240>: lui a0,0x3
0x000000fff62361c4 <+7244>: beq a4,a0,0xfff62369e0 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+9320>
0x000000fff62361c8 <+7248>: li a0,-1
0x000000fff62361cc <+7252>: dsll32 a0,a0,0xf
0x000000fff62361d0 <+7256>: xor a0,a2,a0
0x000000fff62361d4 <+7260>: sd a0,24(s8)
0x000000fff62361d8 <+7264>: beqz a3,0xfff6236954 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+9180>
0x000000fff62361dc <+7268>: lui a0,0x3
0x000000fff62361e0 <+7272>: dsrl32 a1,v0,0x0
0x000000fff62361e4 <+7276>: beq a1,a0,0xfff62369b8 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+9280>
0x000000fff62361e8 <+7280>: li a0,-1
0x000000fff62361ec <+7284>: dsll32 a0,a0,0xf
0x000000fff62361f0 <+7288>: xor v0,v0,a0
0x000000fff62361f4 <+7292>: dmtc1 v0,$f0
0x000000fff62361f8 <+7296>: ldc1 $f2,24(s8)
0x000000fff62361fc <+7300>: li a0,-1
0x000000fff6236200 <+7304>: dsll32 a0,a0,0xf
0x000000fff6236204 <+7308>: mul.d $f2,$f2,$f0
0x000000fff6236208 <+7312>: dmfc1 v0,$f2
0x000000fff623620c <+7316>: xor a0,v0,a0
0x000000fff6236210 <+7320>: lw v0,16(s6)
0x000000fff6236214 <+7324>: bnez v0,0xfff62349f0 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+1144>
---Type <return> to continue, or q <return> to quit---
0x000000fff6236218 <+7328>: nop
0x000000fff623621c <+7332>: lwu v0,24(s4)
0x000000fff6236220 <+7336>: lwu a1,28(s4)
0x000000fff6236224 <+7340>: dsll v0,v0,0x3
0x000000fff6236228 <+7344>: gsldx v0,0(s5,v0)
0x000000fff623622c <+7348>: dsll a1,a1,0x3
0x000000fff6236230 <+7352>: b 0xfff6234bb0 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+1592>
0x000000fff6236234 <+7356>: gssdx a0,0(v0,a1)
0x000000fff6236238 <+7360>: lwu a0,8(s4)
0x000000fff623623c <+7364>: lwu v0,12(s4)
0x000000fff6236240 <+7368>: dsll v1,a0,0x3
0x000000fff6236244 <+7372>: gsldx a5,0(s5,v1)
0x000000fff6236248 <+7376>: dsll v0,v0,0x3
=> 0x000000fff623624c <+7380>: b 0xfff6234a00 <_ZN3QV44Moth3VME3runEPNS_15ExecutionEngineEPKhPPPv+1160>
0x000000fff6236250 <+7384>: gsldx v0,0(a5,v0) 0x000000fff623624c 处是跳转指令,根据mips延迟槽的特点故此处是触发非法指令的指令
0x000000fff6236254 <+7388>: nop
0x000000fff6236258 <+7392>: lwu v0,8(s4)
0x000000fff623625c <+7396>: move a0,s6
0x000000fff6236260 <+7400>: daddiu a1,s4,16
0x000000fff6236264 <+7404>: lwu s7,12(s4)
0x000000fff6236268 <+7408>: ld t9,-1712(gp)
0x000000fff623626c <+7412>: dsll v0,v0,0x3
0x000000fff6236270 <+7416>: dsll s7,s7,0x3
0x000000fff6236274 <+7420>: sd a1,32(s8)
通过查看寄存器cause
的值可以知道cause
寄存器的BD
位被置1,说明异常发生在转移指令的延迟槽中。另外通过解码cause
寄存器中可知ExcCode
值为4. 查表《see mips run》可知异常码4对应的是: (取数、取指或者存数时)地址错误:这要么是在用户态试图存取kuseg以外的空间,或者是试图从未对齐的地址读取一个双字、字或者半字。
寄存器pc
则记录了发生异常时的地址。
badvaddr
这个寄存器保存引发异常的地址;*在发生MMU相关的异常时、在用户程序试图访问kuseg以外的地址时、或者地址没有正确对齐时,该寄存器被设置。*此处的值与a5
的值相同。 正好与gsldx v0,0(a5,v0)
相对应。
ls2k
是mips64
架构,并且运行的nk5操作系统是64位系统,故对齐要求为8字节对齐。而此时a5
的值为ffd412347c
,从而触发非对齐访问。
通过上面的分析找到了产生非法指令的语句是gsldx v0,0(a5,v0)
,一看gsldx
指令并非mips
体系的标准指令,而是龙芯自己扩展的非对齐访问指令。由于mips对非齐访问指令拥有专利权,所以龙芯自己实现一套以绕开专利问题。
自定义指令需要在binuntils
添加实现或是在内核中做指令模拟支持才能被识别。由于中标自带3.10
能正常,而自移植的4.19
的有问题,显然问题出在内核。经查发现内核中没有实现gsldx
指令。
之后在linux 源码树[1]上找到了相应的代码。
在这里面龙梦的源码树[2]更是有针对linux 4.19
的相关代码,将补丁取下来之后给内核打上,重新编译之后一切正常。
[1] https://git.kernel.org/pub/scm/linux/kernel/git/mips/linux.git/commit/?id=f83e4f9896eff614d0f2547a561fa5f39f9cddde
[2] http://dev.lemote.com:8000/cgit/?p=linux-official.git;a=commit;h=09156ac5d92e7cd9729bf7accca1cfd0fc68905b