[java]排查服务器cpu最高的线程

1、通过linux命令排查

1.1、使用top命令定位异常的线程

top

top - 11:08:08 up 3 days, 23:43,  1 user,  load average: 0.00, 0.01, 0.05
Tasks:  76 total,   1 running,  75 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  0.2 sy,  0.0 ni, 99.5 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  3880928 total,  2760024 free,   251708 used,   869196 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  3391008 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                               
 1112 root      10 -10  125012  12636   9812 S   1.0  0.3  56:08.08 AliYunDun                                                                                                             
 2717 root      20   0 3673436 175600  14836 S   0.3  4.5   4:42.16 java 

1.2、查看异常的线程

top -H -p pid

[root@xxfs bin]# top -H -p 2717
top - 08:56:11 up 21:31,  1 user,  load average: 0.02, 0.04, 0.05
Threads:  39 total,   0 running,  39 sleeping,   0 stopped,   0 zombie
%Cpu(s):  3.2 us,  0.0 sy,  0.0 ni, 96.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  3880928 total,  3146892 free,   158104 used,   575932 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  3488932 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                                                                                
 2717 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.00 java                                                                                                                   
 2718 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.90 java                                                                                                                   
 2719 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.03 java                                                                                                                   
 2720 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.02 java                                                                                                                   
 2721 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.01 java                                                                                                                   
 2722 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.00 java                                                                                                                   
 2723 root      20   0 3625184  82100  13576 S  0.0  2.1   0:00.00 java  

1.3、将异常线程转化为16进制

printf “%x\n” pid

[root@xxfs bin]# printf "%x\n" 2718
a9e

1.4、定位异常的代码

jstack 进程号|grep 线程号（16进制异常线程号）-A90

[root@xxfs bin]# jstack 2717 | grep a9e -A90
"main" #1 prio=5 os_prio=0 tid=0x00007f06d000b000 nid=0xa9e runnable [0x00007f06d8d67000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
	at java.net.ServerSocket.implAccept(ServerSocket.java:545)
	at java.net.ServerSocket.accept(ServerSocket.java:513)
	at org.apache.catalina.core.StandardServer.await(StandardServer.java:447)
	at org.apache.catalina.startup.Catalina.await(Catalina.java:776)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:722)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:343)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:474)

"VM Thread" os_prio=0 tid=0x00007f06d00ce800 nid=0xaa6 runnable 

"Gang worker#0 (Parallel GC Threads)" os_prio=0 tid=0x00007f06d0022000 nid=0xa9f runnable 

"Gang worker#1 (Parallel GC Threads)" os_prio=0 tid=0x00007f06d0023800 nid=0xaa0 runnable 

"G1 Main Concurrent Mark GC Thread" os_prio=0 tid=0x00007f06d003b000 nid=0xaa4 runnable 

"Gang worker#0 (G1 Parallel Marking Threads)" os_prio=0 tid=0x00007f06d003d000 nid=0xaa5 runnable 

"G1 Concurrent Refinement Thread#0" os_prio=0 tid=0x00007f06d002a000 nid=0xaa3 runnable 

"G1 Concurrent Refinement Thread#1" os_prio=0 tid=0x00007f06d0028000 nid=0xaa2 runnable 

"G1 Concurrent Refinement Thread#2" os_prio=0 tid=0x00007f06d0026000 nid=0xaa1 runnable 

"VM Periodic Task Thread" os_prio=0 tid=0x00007f06d0278000 nid=0xab2 waiting on condition 

JNI global references: 234

2、通过arthas排查

2.1 下载arthas

https://alibaba.github.io/arthas/download.html#id2

2.2 通过arthas进入某程序

java -jar arthas-boot.jar  pid

2.3 查看cpu耗时最高的线程

[arthas@2717]$ thread -n 1
"as-command-execute-daemon" Id=44 cpuUsage=100% RUNNABLE
    at sun.management.ThreadImpl.dumpThreads0(Native Method)
    at sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:448)
    at com.taobao.arthas.core.command.monitor200.ThreadCommand.processTopBusyThreads(ThreadCommand.java:194)
    at com.taobao.arthas.core.command.monitor200.ThreadCommand.process(ThreadCommand.java:115)
    at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
    at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
    at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
    at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
    at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:372)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)


Affect(row-cnt:0) cost in 104 ms.