项目运行一段时间后会自动假死,使用jstack/jmap命令,服务恢复正常

求大神帮忙!!
springcloud&sprinboot项目运行一段时间后会自动假死,使用jmap命令,服务恢复正常
最近发现项目中的B服务经常莫名假死,就是端口正常开放,cpu处于正常情况,从服务日志中可以看到B服务还可以处理C服务的请求,但是B服务无法响应A服务的请求,使用jstack命令,如下:

2019-07-07 22:09:32
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode):

"Attach Listener" #222 daemon prio=9 os_prio=0 tid=0x00007f6d8800b000 nid=0x7a1e waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"http-nio-6010-exec-160" #221 daemon prio=5 os_prio=0 tid=0x00007f6cec088800 nid=0x7956 waiting on condition [0x00007f6cb6d59000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c7fbcee0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2083)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
        at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-6010-exec-159" #220 daemon prio=5 os_prio=0 tid=0x00007f6ce00b9000 nid=0x77e2 waiting on condition [0x00007f6cb6dda000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c7fbcee0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2083)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
        at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

使用jmap命令之后,服务自动恢复,

Attaching to process ID 29499, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.181-b13

using thread-local object allocation.
Parallel GC with 8 thread(s)

Heap Configuration:
   MinHeapFreeRatio         = 0
   MaxHeapFreeRatio         = 100
   MaxHeapSize              = 4294967296 (4096.0MB)
   NewSize                  = 134217728 (128.0MB)
   MaxNewSize               = 134217728 (128.0MB)
   OldSize                  = 2013265920 (1920.0MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 536870912 (512.0MB)
   CompressedClassSpaceSize = 1065353216 (1016.0MB)
   MaxMetaspaceSize         = 1073741824 (1024.0MB)
   G1HeapRegionSize         = 0 (0.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 132120576 (126.0MB)
   used     = 56769088 (54.13922119140625MB)
   free     = 75351488 (71.86077880859375MB)
   42.96763586619544% used
From Space:
   capacity = 1048576 (1.0MB)
   used     = 425984 (0.40625MB)
   free     = 622592 (0.59375MB)
   40.625% used
To Space:
   capacity = 1048576 (1.0MB)
   used     = 0 (0.0MB)
   free     = 1048576 (1.0MB
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 4
    评论
好的,我可以为您提供一个实现这个功能的Python脚本。但是,需要先确认以下几点: 1. 您的Prometheus已经配置好了告警规则,并能够正确触发告警。 2. 您的服务器上已经安装了ssh,并且您已经拥有了ssh连接到服务器的权限。 3. 您需要在本地生成一个SSH私钥,并将其添加到服务器的authorized_keys中,以避免每次触发告警时都需要手动输入密码。 如果上述条件都满足了,那么我们可以开始编写Python脚本了。以下是一个示例脚本: ```python import subprocess # 定义需要连接的服务器列表 servers = { 'server1': '192.168.1.1', 'server2': '192.168.1.2', 'server3': '192.168.1.3' } # 定义需要执行的命令列表 commands = [ 'jstack -l', 'jmap -dump:format=b,file=/tmp/dump.bin' ] # 定义SSH连接命令 ssh_cmd = 'ssh -i ~/.ssh/id_rsa {} "{}"' # 遍历服务器列表,执行命令 for name, ip in servers.items(): for command in commands: # 构造SSH连接命令 full_cmd = ssh_cmd.format(ip, command) # 执行SSH连接命令 result = subprocess.run(full_cmd, shell=True, capture_output=True) # 将结果输出到控制台 print(f'{name} - {command}\n{result.stdout.decode()}') ``` 这个脚本会遍历定义的服务器列表,依次连接到每个服务器,并执行定义的命令列表中的命令。执行结果会输出到控制台。 您可以根据实际情况修改服务器列表和命令列表,并根据需要添加更多的命令。如果您想将结果保存到文件中,可以使用Python的文件操作相关函数。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 4
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值