查找线上java程序占用服务器cpu100% 原因或者找出程序中的死锁

lichaofan1

已于 2022-06-28 15:41:51 修改

阅读量597

点赞数

分类专栏： JVM 文章标签： java jvm 开发语言

于 2022-06-24 12:09:51 首次发布

本文链接：https://blog.csdn.net/lichaofan1/article/details/125440307

版权

JVM 专栏收录该内容

10 篇文章 0 订阅

订阅专栏

查找线上java程序占用服务器cpu100%

服务在运行中，导致服务器cpu 100%的问题，可能存在原因是while循环，死循环了，一直占有cpu。也有可能是某个线程在不停的在向堆栈空间插入对象，我们可以通过jstack命令来找到可能存在问题的代码块，来快速的定位问题。

首先通过top命令找到cpu占据较高的java程序，或者可直接使用jps命令找到对应的java进程

在这里插入图片描述

这里可以看到cpu占据较高的java进程。注意：这里是进程不是线程。

使用命令top -Hp ，显示你的java进程的内存情况，pid是你的java进程号，比如310147

在这里插入图片描述

liunx下进程的pid是十进制的，因为java线程栈文件中的线程id是十六进制，所以需要转换一下。十进制转十六进制的命令是
printf “%x\n” 312246，
在这里插入图片描述

这里的4c3b6就是上面转换的十六进制的pid。

执行 jstack 进程pid 或者jstack 进程pid|grep -A 10 pid(十六进制的线程pid)，得到线程堆栈信息中 4BE57这个线程所在行的后面10行，从堆栈中可以发现导致cpu飙高的调用方法

在这里插入图片描述

找出程序中的死锁

jstack pid


[root@iZ2zehycnr2ydxty3aokeaZ ~]# jstack 312772
2022-06-24 14:40:05
Full thread dump Java HotSpot(TM) 64-Bit Server VM (18.0.1.1+2-6 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f65d8001df0, length=16, elements={
0x00007f66080ab2f0, 0x00007f66080ac760, 0x00007f66080b6300, 0x00007f66080b7670,
0x00007f66080b8a40, 0x00007f66080ba3b0, 0x00007f66080bb8a0, 0x00007f66080bccd0,
0x00007f66080c4360, 0x00007f66080c7300, 0x00007f66080ce1e0, 0x00007f66080d0400,
0x00007f66080d1480, 0x00007f66080d2630, 0x00007f6608025850, 0x00007f65d8000e70
}

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.31ms elapsed=155.46s tid=0x00007f66080ab2f0 nid=312775 waiting on condition  [0x00007f660e971000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@18.0.1.1/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@18.0.1.1/Reference.java:253)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@18.0.1.1/Reference.java:215)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=0.62ms elapsed=155.46s tid=0x00007f66080ac760 nid=312776 in Object.wait()  [0x00007f660e870000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@18.0.1.1/Native Method)
	- waiting on <0x00000000e50027d8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@18.0.1.1/ReferenceQueue.java:155)
	- locked <0x00000000e50027d8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@18.0.1.1/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@18.0.1.1/Finalizer.java:183)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.77ms elapsed=155.45s tid=0x00007f66080b6300 nid=312777 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.26ms elapsed=155.45s tid=0x00007f66080b7670 nid=312778 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Monitor Deflation Thread" #6 daemon prio=9 os_prio=0 cpu=4.43ms elapsed=155.45s tid=0x00007f66080b8a40 nid=312779 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=7.41ms elapsed=155.45s tid=0x00007f66080ba3b0 nid=312780 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #8 daemon prio=9 os_prio=0 cpu=15.53ms elapsed=155.45s tid=0x00007f66080bb8a0 nid=312781 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #9 daemon prio=9 os_prio=0 cpu=0.19ms elapsed=155.45s tid=0x00007f66080bccd0 nid=312782 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Notification Thread" #10 daemon prio=9 os_prio=0 cpu=0.16ms elapsed=155.44s tid=0x00007f66080c4360 nid=312783 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #11 daemon prio=8 os_prio=0 cpu=0.45ms elapsed=155.44s tid=0x00007f66080c7300 nid=312785 in Object.wait()  [0x00007f660dc9b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@18.0.1.1/Native Method)
	- waiting on <0x00000000e5017be0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@18.0.1.1/ReferenceQueue.java:155)
	- locked <0x00000000e5017be0> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@18.0.1.1/CleanerImpl.java:140)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)
	at jdk.internal.misc.InnocuousThread.run(java.base@18.0.1.1/InnocuousThread.java:162)

"Thread-0" #12 prio=5 os_prio=0 cpu=153606.13ms elapsed=155.42s tid=0x00007f66080ce1e0 nid=312786 runnable  [0x00007f660db9a000]
   java.lang.Thread.State: RUNNABLE
	at com.lcf.jvm.ArthasTest.lambda$cpuhigh$1(ArthasTest.java:39)
	at com.lcf.jvm.ArthasTest$$Lambda$1/0x0000000800c009f0.run(Unknown Source)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)

"Thread-1" #13 prio=5 os_prio=0 cpu=4.23ms elapsed=155.41s tid=0x00007f66080d0400 nid=312787 waiting for monitor entry  [0x00007f660da99000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at com.lcf.jvm.ArthasTest.lambda$deadthread$2(ArthasTest.java:63)
	- waiting to lock <0x00000000e5053970> (a java.lang.Object)
	- locked <0x00000000e5053960> (a java.lang.Object)
	at com.lcf.jvm.ArthasTest$$Lambda$2/0x0000000800c00bf8.run(Unknown Source)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)

"Thread-2" #14 prio=5 os_prio=0 cpu=4.19ms elapsed=155.41s tid=0x00007f66080d1480 nid=312788 waiting for monitor entry  [0x00007f660d998000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at com.lcf.jvm.ArthasTest.lambda$deadthread$3(ArthasTest.java:78)
	- waiting to lock <0x00000000e5053960> (a java.lang.Object)
	- locked <0x00000000e5053970> (a java.lang.Object)
	at com.lcf.jvm.ArthasTest$$Lambda$3/0x0000000800c01800.run(Unknown Source)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)

"Thread-3" #15 prio=5 os_prio=0 cpu=14.91ms elapsed=155.41s tid=0x00007f66080d2630 nid=312789 waiting on condition  [0x00007f660d897000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@18.0.1.1/Native Method)
	at com.lcf.jvm.ArthasTest.lambda$addhashsetthread$0(ArthasTest.java:28)
	at com.lcf.jvm.ArthasTest$$Lambda$4/0x0000000800c01a10.run(Unknown Source)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)

"DestroyJavaVM" #16 prio=5 os_prio=0 cpu=65.87ms elapsed=155.41s tid=0x00007f6608025850 nid=312773 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" #17 daemon prio=9 os_prio=0 cpu=0.29ms elapsed=0.10s tid=0x00007f65d8000e70 nid=312849 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=5.10ms elapsed=155.46s tid=0x00007f66080a8270 nid=312774 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=82.38ms elapsed=155.44s tid=0x00007f66080c5d70 nid=312784 waiting on condition  

JNI global refs: 4, weak refs: 0


Found one Java-level deadlock:
=============================
"Thread-1":
  waiting to lock monitor 0x00007f65b0001860 (object 0x00000000e5053970, a java.lang.Object),
  which is held by "Thread-2"

"Thread-2":
  waiting to lock monitor 0x00005635a0cc53c0 (object 0x00000000e5053960, a java.lang.Object),
  which is held by "Thread-1"

Java stack information for the threads listed above:
===================================================
"Thread-1":
	at com.lcf.jvm.ArthasTest.lambda$deadthread$2(ArthasTest.java:63)
	- waiting to lock <0x00000000e5053970> (a java.lang.Object)
	- locked <0x00000000e5053960> (a java.lang.Object)
	at com.lcf.jvm.ArthasTest$$Lambda$2/0x0000000800c00bf8.run(Unknown Source)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)
"Thread-2":
	at com.lcf.jvm.ArthasTest.lambda$deadthread$3(ArthasTest.java:78)
	- waiting to lock <0x00000000e5053960> (a java.lang.Object)
	- locked <0x00000000e5053970> (a java.lang.Object)
	at com.lcf.jvm.ArthasTest$$Lambda$3/0x0000000800c01800.run(Unknown Source)
	at java.lang.Thread.run(java.base@18.0.1.1/Thread.java:833)

Found 1 deadlock.