几种OOM异常分析

OOM为out of memory的简称,称之为内存溢出。

程序中常见的打印有如下几类:

一:

如图:


Java应用程序在启动时会指定所需要的内存大小,其主要被分割成两个不同的部分,分别为Head space(堆空间-Xmx指定)和Permegen(永久代-XX:MaxPermSize指定),

通常来说,造成如上图异常的基本上程序代码问题而造成的内存泄露。这种异常,通过dump+EMA可以轻松定位。(EMA虽功能强大,但对机器性能内存要求极高)


二:

Java.lang.OutOfMemeoryError:GC overhead limit exceeded

如上异常,即程序在垃圾回收上花费了98%的时间,却收集不回2%的空间,通常这样的异常伴随着CPU的冲高。定位方法同上。


三:

Java.lang.OutOfMemoryError: PermGen space(JAVA8引入了Metaspace区域)

永久代内存被耗尽,永久代的作用是存储每个类的信息,如类加载器引用、运行池常量池、字段数据、方法数据、方法代码、方法字节码等。基本可以推断PermGen占用大小取决于被加载的数量以及类的大小。定位方法同上。


还有很多OOM异常,甚至会触发操作系统的OOM killer去杀掉其它进程。


四:

本节主要讨论下面一种OOM,如图


产生这种异常的原因是由于系统在不停地创建大量的线程,且不进行释放。系统的内存是有限的,分配给JAVA应用的程序也是有限的,系统自身能允许创建的最大线程数计算规则:

(MaxProcessMemory-JVMMemory-ReservedOsMemory)/ThreadStackSize

其中

MaxProcessMemory:指的是一个进程的最大内存

JVMMemory :JVM内存

ReservedOsMemory:保留的操作系统内存

ThreadStackSize:线程栈的大小

从公式中可以得出结论,系统可创建线程数量与分配给JVM内存大小成反比。


在java程序中,创建一个线程,虚拟机会在JVM内存创建一个Thread对象同时创建一个操作系统线程,且系统线程的内存不占用JVMMemory,而占用系统中剩下的内存,即(MaxProcessMemory - JVMMemory - ReservedOsMemory)。

结论很明显:程序创建的线程不可能无限大。


先讨论第一种情况,即经jstack或dump后,线程的数量确在系统要求的阀值内,报上面异常,该如何?

1.参考之前的参数,可以修改两个变量JVMMemory和ThreadStackSize来满足更多线程创建的要求。

-Xss1024k -Xms4096m -Xmx4096m

2.查看是否操作系统限制了可创建线程数量

执行ulimit -u 可以查看当前用户可创建线程量,如果不满足要求,可以通过修改配置文件调整其大小:

相关配置文件在etc/security/limit.d/XX-nproc.conf中


由于上面配置不合理造成而产生异常,个人认为应属小概率事件。


第二种情况,是程序中存在线程泄露,代码本身有问题,在某些场景,条件下存在线程不断创建而不销毁的BUG。

先看一段代码(取自业务真实代码稍加改写):

package com.zte.sunquan.demo.netty.server;
import com.google.common.collect.Maps;
import java.util.Map;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MainController {
    private ExecutorService executor = Executors.newSingleThreadExecutor();
    private static Map<String, MainController> map = Maps.newConcurrentMap();
    public void submitTask() {
        for (int i = 0; i < 10; i++) {
            executor.submit(new Runnable() {
                @Override
                public void run() {
       System.out.println(Thread.currentThread().getName() + "正处理");
                }
            });
        }
    }
    public static void main(String[] args) throws InterruptedException {
        while (true) {
            MainController ct1 = new MainController();
            map.put("1", ct1);
            ct1.submitTask();
            map.remove("1");
        }
    }
}

如上表的程序就存在线程泄露,开发人员误以为从map中移除,对象就可以释放,最终交由GC回收,但其实在创建的对象中创建的线程池,由于未关闭,执行完任务后,进入waiting状态,是不会释放的,随着程序运行,线程会越积越多,最终导致OOM的异常。

又或者将main函数,改为一个监听入口,在一般甚至短暂的压力测试中,虽然线程较多,但系统仍可以正常进行,误以为系统运行正常,但大业务、多节点、网络震荡、长时间,商用等能够大量触发该监听的场景中,这类问题才以最终kill系统进程暴露出来。

此外,如上表中的代码,线程池创建的进程,采用Executors中DefaultThreadFactory提供的pool-XXX-thread-XXX命名规则,诚然可以通过jstack打印线程调用栈,但如下面的打印:

.....
"pool-2577-thread-1" #8681 prio=5 os_prio=0 tid=0x00007f8ea4de7800 nid=0x6a0b waiting on condition [0x00007f8cae227000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x000000057136f4b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"pool-2576-thread-1" #8680 prio=5 os_prio=0 tid=0x00007f8ea4de6000 nid=0x6a0a waiting on condition [0x00007f8cae72c000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x000000057136fa20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
......

完整打印一直是从pool-1-thread-1到pool-5406-thread-1,很明显代码一直在创建线程池,但从打印输出,却无法分析出罪魁祸首是谁?接下来自然,会去系统日志中搜索pool-开头的线程打印,期望能找到线索,遗憾的是,什么信息也没有。此时问题定位,卡壳了。但可以肯定,代码肯定有前面例子中类似的写法。此时,走查代码,不失为一个好办法

线索就此断掉了。。。。。

但考虑到是使用JDK并发包提供的功能,那在JDK中Executors类中,创建线程时,增加打印,强制将调用栈的信息打印出来,是否可以找到蛛丝马迹?

确定JDK版本--->获取源码---->修改代码--->编译-------->合入rt.jar ------->复现

我们将最终信息输入到了var/log目录下,需要提前将该目录的权限开放chmod 777 -R


输出日志中,对一些连续创建的线程,观察打印信息,幸运女神出现了,OpticalTopologyAdapter,抓到你了。问题解决!

这种问题的出现,归根到底,是没有对线程池进行管理,开发人员对于线程池的滥用,影响了程序稳定性。

修改的源文件如下:

package java.util.concurrent;

import sun.security.util.SecurityConstants;

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.security.AccessControlContext;
import java.security.AccessControlException;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.security.PrivilegedActionException;
import java.security.PrivilegedExceptionAction;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class Executors {

   
    public static ExecutorService newFixedThreadPool(int nThreads) {
        StackTraceElement[] elements = Thread.currentThread().getStackTrace();
        recordThreadStackInfo("SQ:FixThreadPool: " + Thread.currentThread().getName() + nThreads, elements);
        return new ThreadPoolExecutor(nThreads, nThreads,
                0L, TimeUnit.MILLISECONDS,
                new LinkedBlockingQueue<Runnable>());
    }

   
    public static ExecutorService newWorkStealingPool(int parallelism) {
        return new ForkJoinPool
                (parallelism,
                        ForkJoinPool.defaultForkJoinWorkerThreadFactory,
                        null, true);
    }

   
    public static ExecutorService newWorkStealingPool() {
        return new ForkJoinPool
                (Runtime.getRuntime().availableProcessors(),
                        ForkJoinPool.defaultForkJoinWorkerThreadFactory,
                        null, true);
    }

   
    public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory) {
        StackTraceElement[] elements = Thread.currentThread().getStackTrace();
        recordThreadStackInfo("SQ2:FixThreadPool: " + Thread.currentThread().getName() + nThreads, elements);
        return new ThreadPoolExecutor(nThreads, nThreads,
                0L, TimeUnit.MILLISECONDS,
                new LinkedBlockingQueue<Runnable>(),
                threadFactory);
    }

    
    public static ExecutorService newSingleThreadExecutor() {
        StackTraceElement[] elements = Thread.currentThread().getStackTrace();
        recordThreadStackInfo("SQ1:SingleThread: " + Thread.currentThread().getName(), elements);
        return new FinalizableDelegatedExecutorService
                (new ThreadPoolExecutor(1, 1,
                        0L, TimeUnit.MILLISECONDS,
                        new LinkedBlockingQueue<Runnable>()));
    }

    
    public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory) {
        StackTraceElement[] elements = Thread.currentThread().getStackTrace();
        recordThreadStackInfo("SQ2:SingleThread: " + Thread.currentThread().getName(), elements);
        return new FinalizableDelegatedExecutorService
                (new ThreadPoolExecutor(1, 1,
                        0L, TimeUnit.MILLISECONDS,
                        new LinkedBlockingQueue<Runnable>(),
                        threadFactory));
    }

    
    public static ExecutorService newCachedThreadPool() {
        return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                60L, TimeUnit.SECONDS,
                new SynchronousQueue<Runnable>());
    }

    
    public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory) {
        return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                60L, TimeUnit.SECONDS,
                new SynchronousQueue<Runnable>(),
                threadFactory);
    }

    
    public static ScheduledExecutorService newSingleThreadScheduledExecutor() {
        return new DelegatedScheduledExecutorService
                (new ScheduledThreadPoolExecutor(1));
    }

    
    public static ScheduledExecutorService newSingleThreadScheduledExecutor(ThreadFactory threadFactory) {
        return new DelegatedScheduledExecutorService
                (new ScheduledThreadPoolExecutor(1, threadFactory));
    }

    
    public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
        return new ScheduledThreadPoolExecutor(corePoolSize);
    }

   
    public static ScheduledExecutorService newScheduledThreadPool(
            int corePoolSize, ThreadFactory threadFactory) {
        return new ScheduledThreadPoolExecutor(corePoolSize, threadFactory);
    }

    
    public static ExecutorService unconfigurableExecutorService(ExecutorService executor) {
        if (executor == null)
            throw new NullPointerException();
        return new DelegatedExecutorService(executor);
    }

    
    public static ScheduledExecutorService unconfigurableScheduledExecutorService(ScheduledExecutorService executor) {
        if (executor == null)
            throw new NullPointerException();
        return new DelegatedScheduledExecutorService(executor);
    }

    
    public static ThreadFactory defaultThreadFactory() {
        return new DefaultThreadFactory();
    }

    
    public static ThreadFactory privilegedThreadFactory() {
        return new PrivilegedThreadFactory();
    }

    
    public static <T> Callable<T> callable(Runnable task, T result) {
        if (task == null)
            throw new NullPointerException();
        return new RunnableAdapter<T>(task, result);
    }

   
    public static Callable<Object> callable(Runnable task) {
        if (task == null)
            throw new NullPointerException();
        return new RunnableAdapter<Object>(task, null);
    }

    
    public static Callable<Object> callable(final PrivilegedAction<?> action) {
        if (action == null)
            throw new NullPointerException();
        return new Callable<Object>() {
            public Object call() {
                return action.run();
            }
        };
    }

   
    public static Callable<Object> callable(final PrivilegedExceptionAction<?> action) {
        if (action == null)
            throw new NullPointerException();
        return new Callable<Object>() {
            public Object call() throws Exception {
                return action.run();
            }
        };
    }

    
    public static <T> Callable<T> privilegedCallable(Callable<T> callable) {
        if (callable == null)
            throw new NullPointerException();
        return new PrivilegedCallable<T>(callable);
    }

    
    public static <T> Callable<T> privilegedCallableUsingCurrentClassLoader(Callable<T> callable) {
        if (callable == null)
            throw new NullPointerException();
        return new PrivilegedCallableUsingCurrentClassLoader<T>(callable);
    }

    // Non-public classes supporting the public methods

    
    static final class RunnableAdapter<T> implements Callable<T> {
        final Runnable task;
        final T result;

        RunnableAdapter(Runnable task, T result) {
            this.task = task;
            this.result = result;
        }

        public T call() {
            task.run();
            return result;
        }
    }

    
    static final class PrivilegedCallable<T> implements Callable<T> {
        private final Callable<T> task;
        private final AccessControlContext acc;

        PrivilegedCallable(Callable<T> task) {
            this.task = task;
            this.acc = AccessController.getContext();
        }

        public T call() throws Exception {
            try {
                return AccessController.doPrivileged(
                        new PrivilegedExceptionAction<T>() {
                            public T run() throws Exception {
                                return task.call();
                            }
                        }, acc);
            } catch (PrivilegedActionException e) {
                throw e.getException();
            }
        }
    }

   
    static final class PrivilegedCallableUsingCurrentClassLoader<T> implements Callable<T> {
        private final Callable<T> task;
        private final AccessControlContext acc;
        private final ClassLoader ccl;

        PrivilegedCallableUsingCurrentClassLoader(Callable<T> task) {
            SecurityManager sm = System.getSecurityManager();
            if (sm != null) {
                // Calls to getContextClassLoader from this class
                // never trigger a security check, but we check
                // whether our callers have this permission anyways.
                sm.checkPermission(SecurityConstants.GET_CLASSLOADER_PERMISSION);

                // Whether setContextClassLoader turns out to be necessary
                // or not, we fail fast if permission is not available.
                sm.checkPermission(new RuntimePermission("setContextClassLoader"));
            }
            this.task = task;
            this.acc = AccessController.getContext();
            this.ccl = Thread.currentThread().getContextClassLoader();
        }

        public T call() throws Exception {
            try {
                return AccessController.doPrivileged(
                        new PrivilegedExceptionAction<T>() {
                            public T run() throws Exception {
                                Thread t = Thread.currentThread();
                                ClassLoader cl = t.getContextClassLoader();
                                if (ccl == cl) {
                                    return task.call();
                                } else {
                                    t.setContextClassLoader(ccl);
                                    try {
                                        return task.call();
                                    } finally {
                                        t.setContextClassLoader(cl);
                                    }
                                }
                            }
                        }, acc);
            } catch (PrivilegedActionException e) {
                throw e.getException();
            }
        }
    }


    private static void recordThreadStackInfo(String headInfo, StackTraceElement[] elements) {
        StringBuffer buf = new StringBuffer(headInfo);
        for (int i = 0; i < elements.length; i++) {
            buf.append("\n    "
                    + elements[i].getClassName()
                    + "."
                    + elements[i].getMethodName()
                    + "("
                    + elements[i].getFileName()
                    + ":"
                    + elements[i].getLineNumber()
                    + ")\n");
        }
        recordInfo(buf.toString());
    }

    private static void recordInfo(String info) {
        File f = new File("/var/log/threadsInfo.txt");
        if (!f.exists()) {
            try {
                f.createNewFile();
            } catch (IOException e1) {
                e1.printStackTrace();
            }
        }
        FileWriter fileWriter = null;
        try {
            fileWriter = new FileWriter(f, true);
            fileWriter.write(info);
            fileWriter.flush();
        } catch (IOException e1) {
            e1.printStackTrace();
        } finally {
            if (fileWriter != null)
                try {
                    fileWriter.close();
                } catch (IOException e1) {
                    e1.printStackTrace();
                }
        }
    }

    private static void printStackTrace(String emsg) {
        Exception e = new Exception(emsg);
        e.printStackTrace();
    }

    /**
     * The default thread factory
     */
    static class DefaultThreadFactory implements ThreadFactory {
        private static final AtomicInteger poolNumber = new AtomicInteger(1);
        private final ThreadGroup group;
        private final AtomicInteger threadNumber = new AtomicInteger(1);
        private final String namePrefix;

        DefaultThreadFactory() {
            SecurityManager s = System.getSecurityManager();
            group = (s != null) ? s.getThreadGroup() :
                    Thread.currentThread().getThreadGroup();
            namePrefix = "pool-" +
                    poolNumber.getAndIncrement() +
                    "-thread-";
        }

        public Thread newThread(Runnable r) {
            Thread t = new Thread(group, r,
                    namePrefix + threadNumber.getAndIncrement(),
                    0);
            ///
            String threadNum = namePrefix + threadNumber.get();
            if (poolNumber.get() > 20 && namePrefix.startsWith("pool-")) {
                StackTraceElement[] elements = Thread.currentThread().getStackTrace();
                recordThreadStackInfo("SQ2:SingleThread: " + threadNum + " name:" + t.getName(), elements);
                Exception e = new Exception("SQ newThread Exception");
                e.printStackTrace();
            }
            ///
            if (t.isDaemon())
                t.setDaemon(false);
            if (t.getPriority() != Thread.NORM_PRIORITY)
                t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }
    }

    /**
     * Thread factory capturing access control context and class loader
     */
    static class PrivilegedThreadFactory extends DefaultThreadFactory {
        private final AccessControlContext acc;
        private final ClassLoader ccl;

        PrivilegedThreadFactory() {
            super();
            SecurityManager sm = System.getSecurityManager();
            if (sm != null) {
                // Calls to getContextClassLoader from this class
                // never trigger a security check, but we check
                // whether our callers have this permission anyways.
                sm.checkPermission(SecurityConstants.GET_CLASSLOADER_PERMISSION);

                // Fail fast
                sm.checkPermission(new RuntimePermission("setContextClassLoader"));
            }
            this.acc = AccessController.getContext();
            this.ccl = Thread.currentThread().getContextClassLoader();
        }

        public Thread newThread(final Runnable r) {
            return super.newThread(new Runnable() {
                public void run() {
                    AccessController.doPrivileged(new PrivilegedAction<Void>() {
                        public Void run() {
                            Thread.currentThread().setContextClassLoader(ccl);
                            r.run();
                            return null;
                        }
                    }, acc);
                }
            });
        }
    }

    /**
     * A wrapper class that exposes only the ExecutorService methods
     * of an ExecutorService implementation.
     */
    static class DelegatedExecutorService extends AbstractExecutorService {
        private final ExecutorService e;

        DelegatedExecutorService(ExecutorService executor) {
            e = executor;
        }

        public void execute(Runnable command) {
            e.execute(command);
        }

        public void shutdown() {
            e.shutdown();
        }

        public List<Runnable> shutdownNow() {
            return e.shutdownNow();
        }

        public boolean isShutdown() {
            return e.isShutdown();
        }

        public boolean isTerminated() {
            return e.isTerminated();
        }

        public boolean awaitTermination(long timeout, TimeUnit unit)
                throws InterruptedException {
            return e.awaitTermination(timeout, unit);
        }

        public Future<?> submit(Runnable task) {
            return e.submit(task);
        }

        public <T> Future<T> submit(Callable<T> task) {
            return e.submit(task);
        }

        public <T> Future<T> submit(Runnable task, T result) {
            return e.submit(task, result);
        }

        public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks)
                throws InterruptedException {
            return e.invokeAll(tasks);
        }

        public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks,
                                             long timeout, TimeUnit unit)
                throws InterruptedException {
            return e.invokeAll(tasks, timeout, unit);
        }

        public <T> T invokeAny(Collection<? extends Callable<T>> tasks)
                throws InterruptedException, ExecutionException {
            return e.invokeAny(tasks);
        }

        public <T> T invokeAny(Collection<? extends Callable<T>> tasks,
                               long timeout, TimeUnit unit)
                throws InterruptedException, ExecutionException, TimeoutException {
            return e.invokeAny(tasks, timeout, unit);
        }
    }

    static class FinalizableDelegatedExecutorService
            extends DelegatedExecutorService {
        FinalizableDelegatedExecutorService(ExecutorService executor) {
            super(executor);
        }

        protected void finalize() {
            super.shutdown();
        }
    }

    /**
     * A wrapper class that exposes only the ScheduledExecutorService
     * methods of a ScheduledExecutorService implementation.
     */
    static class DelegatedScheduledExecutorService
            extends DelegatedExecutorService
            implements ScheduledExecutorService {
        private final ScheduledExecutorService e;

        DelegatedScheduledExecutorService(ScheduledExecutorService executor) {
            super(executor);
            e = executor;
            System.out.println("b");
        }

        public ScheduledFuture<?> schedule(Runnable command, long delay, TimeUnit unit) {
            return e.schedule(command, delay, unit);
        }

        public <V> ScheduledFuture<V> schedule(Callable<V> callable, long delay, TimeUnit unit) {
            return e.schedule(callable, delay, unit);
        }

        public ScheduledFuture<?> scheduleAtFixedRate(Runnable command, long initialDelay, long period, TimeUnit unit) {
            return e.scheduleAtFixedRate(command, initialDelay, period, unit);
        }

        public ScheduledFuture<?> scheduleWithFixedDelay(Runnable command, long initialDelay, long delay, TimeUnit unit) {
            return e.scheduleWithFixedDelay(command, initialDelay, delay, unit);
        }
    }

    /**
     * Cannot instantiate.
     */
    private Executors() {
    }
}

因此有了下面的几点建议:

1.如果使用JDK创建线程池,必须指定命名规则,参考


ThreadFactory build=new ThreadFactoryBuilder().setPriority(Thread.NORM_PRIORITY)
                                 .setDaemon(false)
                                 .setNameFormat("SQ-Creat-%d")
                                 .build();
Executors.newSingleThreadScheduledExecutor(build);

但使用上面的方式只能设置线程名,而线程池的数目,是无法判断的。简而言之,即有SQ-Create-1,SQ-Create-2.SQ-Create-3,无法判断出这是一个线程池中三个线程,还是存在于两个线程池中。

其实,更简单的方法,只要实现ThreadFactory,稍等改动,则可以实现。(使用UserThreadFactory强制使用必须传入线程池名)


package com.zte.sunquan.demo.executor;

import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * Created by 10184538 on 2018/1/4.
 */
public class UserThreadFactory implements ThreadFactory {
    private static final AtomicInteger poolNumber = new AtomicInteger(1);
    private final ThreadGroup group;
    private final AtomicInteger threadNumber = new AtomicInteger(1);
    private final String namePrefix;

    private UserThreadFactory(String threadPoolName) {
        SecurityManager s = System.getSecurityManager();
        group = (s != null) ? s.getThreadGroup() :
                Thread.currentThread().getThreadGroup();
        namePrefix = threadPoolName + "-" +
                poolNumber.getAndIncrement() +
                "-thread-";
    }

    public Thread newThread(Runnable r) {
        Thread t = new Thread(group, r,
                namePrefix + threadNumber.getAndIncrement(),
                0);
        if (t.isDaemon())
            t.setDaemon(false);
        if (t.getPriority() != Thread.NORM_PRIORITY)
            t.setPriority(Thread.NORM_PRIORITY);
        return t;
    }

    public static UserThreadFactory build(String poolThreadName) {
        return new UserThreadFactory(poolThreadName);
    }
}
2.略

3.最后简单介绍下ONOS线程池的封装使用(以下参考ONOS1.5.2版本)

Onos中考虑到线程的管理,使用了:

SharedExecutors

SharedExecutorService

SharedScheduledExecutors

ShardScheduleExecutorServce

对JDK的Executors进行了封装,增加了部分管理。同时其特点提供的GroupedThreadFactory支持线程的子父节关系定义管理。如:


@Test
public void test() throws InterruptedException {
    //对每一个事件都要定义其所在的事件组,事件名,并保持相关继承关系
    CountDownLatch countDownLatch = new CountDownLatch(1);
    ExecutorService executorService =
            Executors.newFixedThreadPool(2, getFactory("sqGroup/a/b", "sqThread%d"));
    executorService.execute(() -> {
        System.out.println("parent thread group name: " + Thread.currentThread().getThreadGroup().getParent().getName());
        System.out.println("current thread group name: " + Thread.currentThread().getThreadGroup().getName());
        System.out.println("current thread name: " + Thread.currentThread().getName());
        countDownLatch.countDown();
    });
    countDownLatch.await();
    System.out.println(Thread.currentThread().getThreadGroup().getName());
}

打印:

parent thread group name: sqGroup/a

current thread group name: sqGroup/a/b

current thread name: sqThread0

main



总结:本人故障定位中查到的N多OOM问题,原因不外乎以下3类

1、线程池不管理式滥用

2、本地缓存滥用(如只需要使用Node的id,但将整个Node所有信息都缓存)

3、特殊场景考虑不足(如采用单线程处理复杂业务,环境震荡+多设备下积压任务暴增)


==============利用gc.log进行OOM监控=======================

针对JVM的监听,JDK默认提供了如jconsole、jvisualVM工具都非常好用,本节介绍利用打开gc的日志功能,进行OOM的监控。

首先需要对JAVA程序添加执行参数:

-XX:+PrintGC 输出GC日志

-XX:+PrintGCDetails 输出GC的详细日志

-XX:+PrintGCTimeStamps 输出GC的时间戳(以基准时间的形式)

-XX:+PrintGCDateStamps 输出GC的时间戳(以日期的形式,如 2013-05-04T21:53:59.234+0800)

-XX:+PrintHeapAtGC 在进行GC的前后打印出堆的信息

-Xloggc:../logs/gc.log 日志文件的输出路径

示例参数配置:

-xmx100M -XX:+PrintGCDetails -Xloggc:../logs/gc.log -XX:+PrintGCDetails   -Xmx设置堆内存


输出日志的详细介绍:



总共分配了100M堆内存空间,老年代+年轻代=堆

就堆内存,此次GC共回收了79186-75828=3356=3.279M内存,能回收空间已经非常少了。

同时从PSYoungGen回收10612-7254=3558=3.474M内存

3558-3356=202K,说明有202K的空间在年轻代释放,却传入年老代继续占用

 下一条日志输出:

2018-01-03T10:53:52.281+0800: 11.052: [Full GC (Allocation Failure) [PSYoungGen: 7254K->7254K(24064K)] [ParOldGen: 68574K->68486K(68608K)] 75828K->75740K(92672K), [Metaspace: 3357K->3357K(1056768K)], 0.6010057 secs] [Times: user=2.08 sys=0.00, real=0.60 secs]

68574-68486=88K

75828-75740=88K

此次FullGC 只能释放老年代的88K空间


ps.对于分析gc.log,提供一种图形化工具辅助分析

gcviewer-1.3.6


展开阅读全文
博主设置当前文章不允许评论。

没有更多推荐了,返回首页