java源码分析系列一 线程池Executors

            用了线程池已经有一段时间了,以前只是偶尔看看源码,了解了其中调度策略,没有深入研究。因为平常没有遇见什么问题。但是作为一个程序员要严格要求自己,做到未雨绸缪废话不说了,开始我们的源码之旅！

            相信刚开始用java自带线程池,都是是这样用的,

            ExecutorService threadpol=Executors.newFixedThreadPool(...)

            ExecutorService threadpol=Executors.newCachedThreadPool(...)

            ExecutorService threadpol=Executors.newSingleThreadExecutor(...)

        他们对应的源代码

<!---jdk1.7-->

    public static ExecutorService newFixedThreadPool(int nThreads) {
        return new ThreadPoolExecutor(nThreads, nThreads,
                                      0L, TimeUnit.MILLISECONDS,
                                      new LinkedBlockingQueue<Runnable>());

          

       public static ExecutorService newCachedThreadPool() {
        return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                      60L, TimeUnit.SECONDS,
                                      new SynchronousQueue<Runnable>());
    }

 ThreadPoolExecutor的构造函数

 public ThreadPoolExecutor(int corePoolSize,int maximumPoolSize,  long keepAliveTime, TimeUnit unit,  BlockingQueue<Runnable> workQueue,ThreadFactory threadFactory )

所谓newFixedThreadPool 就是coreSize 和maxSize一样大,然后队列采用的是LinkedBlockingQueue,由于coreSize和maxSize一样大,自然keepAliveTime参数就没意义了

所谓newCachedThreadPool就是线程从0开始然后最大线程是2的32次幂,队列用的是同步队列,同步队列的意思就是不保存任务! 好了,先了解下他们的静态结构

这里用到了BlokingQueue,这是一个阻塞队列。我们这里主要用到他的3个子类,分别是ArrayBlockingQueue LinkedBlockingQueue AsynchronousQueue,他们3个的区别如下图

下面带着任务去解开线程池调度的神秘面纱!

一 线程池如果调度的？

以execute()方法开始分析

  int c = ctl.get();
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        else if (!addWorker(command, false))
            reject(command);

先来分析下第一行,从jdk1.7后,java用了大量的位移操作,读懂这些代码对阅读jdk源码很有好处。下面我们就来分析下,这个逻辑,找到ctl这个变量的初始化地方,原来在类头，为了方便阅读,在源码的每一行都加上了注释

//这个是用一个int来表示workerCount和runState的，其中runState占int的高3位，其它29位为workerCount的值。

 private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));

// 这个变量   29=32-3

    private static final int COUNT_BITS = Integer.SIZE - 3;

/*关于java中移位的小技巧

x<<y 结果等于x乘以2的y次方 取整

x>>y 结果等于x除以2的y次方  取整

**/

//1<<29 -1 结果等于1*2的29次幂-1 等于这是线程池的容量

   private static final int CAPACITY   = (1 << COUNT_BITS) - 1;

//  下面用几个数表示线程池中的状态

  // runState is stored in the high-order bits

//取 -1*2的29次幂的值表示当前线程池在运行状态

//该状态下线程池能接受新任务，并且可以运行队列中的任务

    private static final int RUNNING    = -1 << COUNT_BITS;

//该状态下的线程池不再接受新任务，但仍可以执行队列中的任务

    private static final int SHUTDOWN   =  0 << COUNT_BITS;

//状态下的线程池不再接受新任务，不再执行队列中的任务，而且要中断正在处理的任务

    private static final int STOP       =  1 << COUNT_BITS;

    private static final int TIDYING    =  2 << COUNT_BITS;
    private static final int TERMINATED =  3 << COUNT_BITS;

  
    // Packing and unpacking ctl

//取出runState 也就是高3位

    private static int runStateOf(int c)     { return c & ~CAPACITY; }

//取出 workCount   也就是低29位

    private static int workerCountOf(int c)  { return c & CAPACITY; }

//这个是将高3位和低29位保留到1个int里面

    private static int ctlOf(int rs, int wc) { return rs | wc; }

到这里，我们基本可以知道第一行什么意思了吧！就是获取一个包含线程运行状态和线程池中线程数量的整数!来看第2行,

if (workerCountOf(c) < corePoolSize) {

从这个int中取出线程池中数量,并和coreSize比较,如果小于coreSize则执行addWorker(command, true)；下面看看addWorker方法

 private boolean addWorker(Runnable firstTask, boolean core) {
        retry:
        for (;;) {
            int c = ctl.get();

//取出当前线程池的状态

            int rs = runStateOf(c);

            // Check if queue empty only if necessary.

//如果线程池中的状态不是running则返回添加失败,

//如果线程状态是shutdown,虽然不能添加新任务(firstTask必须为null),但可以从执行队列中的任务（workQueue必须不为空）

            if (rs >= SHUTDOWN &&
                ! (rs == SHUTDOWN &&
                   firstTask == null &&
                   ! workQueue.isEmpty()))
                return false;

            for (;;) {

//判断线程中的数量是不是已经达到最大量

                int wc = workerCountOf(c);
                if (wc >= CAPACITY ||
                    wc >= (core ? corePoolSize : maximumPoolSize))
                    return false;
                if (compareAndIncrementWorkerCount(c))
                    break retry;
                c = ctl.get();  // Re-read ctl
                if (runStateOf(c) != rs)
                    continue retry;
                // else CAS failed due to workerCount change; retry inner loop
            }
        }

//创建新线程 开始......
        boolean workerStarted = false;
        boolean workerAdded = false;
        Worker w = null;
        try {
            final ReentrantLock mainLock = this.mainLock;

//创建一个work对象,根据上面work的类图,可以看到work里面的thread对象,其实是我们对我们runable对象的封装,下面贴出创建线程的逻辑，当然用户可以自定义ThreadFactory，下面是默认的创建方式!看到线程名非常熟悉有没有,有没有恍然大悟的感觉！！

***************插播的代码**********************

        public Thread newThread(Runnable r) {
            Thread t = new Thread(group, r,
                                  namePrefix + threadNumber.getAndIncrement(),
                                  0);
            if (t.isDaemon())
                t.setDaemon(false);
            if (t.getPriority() != Thread.NORM_PRIORITY)
                t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }

***********************************

            w = new Worker(firstTask);

//t 就是通过上面插播代码创建的一个线程

            final Thread t = w.thread;
            if (t != null) {

//因为我们的线程池hashSet是线程不安全的,所以往里面放的时候要加锁

                mainLock.lock();
                try {
                    // Recheck while holding lock.
                    // Back out on ThreadFactory failure or if
                    // shut down before lock acquired.
                    int c = ctl.get();
                    int rs = runStateOf(c);

                    if (rs < SHUTDOWN ||
                        (rs == SHUTDOWN && firstTask == null)) {
                        if (t.isAlive()) // precheck that t is startable
                            throw new IllegalThreadStateException();

、//这是将work放入线程池的代码

                        workers.add(w);
                        int s = workers.size();
                        if (s > largestPoolSize)
                            largestPoolSize = s;
                        workerAdded = true;
                    }
                } finally {
                    mainLock.unlock();
                }

//如果加入成功,则启动这个线程

                if (workerAdded) {
                    t.start();
                    workerStarted = true;
                }
            }
        } finally {
            if (! workerStarted)
                addWorkerFailed(w);
        }
        return workerStarted;
    }

************上面分析到将work线程启动成功了,我们看下启动成功了之后做了什么************************

        /** Delegates main run loop to outer runWorker  */
        public void run() {
            runWorker(this);
        }

final void runWorker(Worker w) {
        Thread wt = Thread.currentThread();
        Runnable task = w.firstTask;
        w.firstTask = null;
        w.unlock(); // allow interrupts

//默认是由于异常引发的退出

        boolean completedAbruptly = true;
        try {

/**

注意这里的逻辑:每一次task执行完之后,就将这个task置为null,这个时候我们强大的垃圾回收器就会把这个对象给回收了

getTask()方法是个阻塞方法,一致等待队列中的任务进来。设计的很巧妙有没有。总的来说就是执行过我们传进来的runable对象之后

当前线程并没有立即结束,而是仍然在while循环中,除非抛出了异常,如果抛出异常,会被捕获并且会通过 afterExecute()方法传递出去,如果我们事先重写了这个方法，

就可以将异常打印出来,否则就异常信息就莫名被吞掉了有没有!

**/

            while (task != null || (task = getTask()) != null) {
                w.lock();
                // If pool is stopping, ensure thread is interrupted;
                // if not, ensure thread is not interrupted.  This
                // requires a recheck in second case to deal with
                // shutdownNow race while clearing interrupt
                if ((runStateAtLeast(ctl.get(), STOP) ||
                     (Thread.interrupted() &&
                      runStateAtLeast(ctl.get(), STOP))) &&
                    !wt.isInterrupted())
                    wt.interrupt();
                try {

/***调入之前执行方法,默认这个方法是空的,我们可以自定义,在执行run方法之前做些操作,比如记录下每个runable对象的执行开始时间,然后结束的时候记录下结束时间,就可以统计每个线程执行的时间

***/

                    beforeExecute(wt, task);
                    Throwable thrown = null;
                    try {

//调用我们传入的runable对象的run方法!!!

                        task.run();
                    } catch (RuntimeException x) {
                        thrown = x; throw x;
                    } catch (Error x) {
                        thrown = x; throw x;
                    } catch (Throwable x) {
                        thrown = x; throw new Error(x);
                    } finally {
                        afterExecute(task, thrown);
                    }
                } finally {
                    task = null;
                    w.completedTasks++;
                    w.unlock();
                }
            }

// false代表不是由于异常引发的线程退出

            completedAbruptly = false;
        } finally {
            processWorkerExit(w, completedAbruptly);
        }
    }

如果我们的业务run方法抛出异常,通过afterExcute传递出去,然后依次进入2个finally块,我们重点看第2个finally块的代码processWorkerExit(w, completedAbruptly);

 private void processWorkerExit(Worker w, boolean completedAbruptly) {
        if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
            decrementWorkerCount();

        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            completedTaskCount += w.completedTasks;
            workers.remove(w);
        } finally {
            mainLock.unlock();
        }

        tryTerminate();

        int c = ctl.get();
        if (runStateLessThan(c, STOP)) {
            if (!completedAbruptly) {
                int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
                if (min == 0 && ! workQueue.isEmpty())
                    min = 1;
                if (workerCountOf(c) >= min)
                    return; // replacement not needed
            }

//如果是异常退出,会重新add一个work线程,只是不指定firstTask

            addWorker(null, false);
        }
    }

 **********************addWorker分析结束***************************************

回到ThreadPoolExecutor的execute()方法,addWorker的过程就是就是新建线程的过程,加入线程线程成功exectue方法就return 了,如果失败了,我们继续往下分析

  int c = ctl.get();

//如果当前线程池中数量小于coreSize,则添加线程

        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }

//线程池在running状态,线程池中数量大于coreSize,往task队列插入

        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }

//队列塞不进去了,就往线程池中增加新线程,以非core的方式添加。如果队列塞不进去了,添加线程也失败了,就调用reject策略。

        else if (!addWorker(command, false))
            reject(command);

reject方法很简单,调用handler的方法,而handler是ThreadPoolExecutor类的成员变量,我们可以注入我们自己的hander。

   /**
     * Invokes the rejected execution handler for the given command.
     * Package-protected for use by ScheduledThreadPoolExecutor.
     */
    final void reject(Runnable command) {
        handler.rejectedExecution(command, this);
    }

总结下:

RejectedExecutionHandler 4中预定义的处理策略

CallerRunsPolicy 当有任务添加到线程池被拒绝时，线程池会将被拒绝的任务添加到”线程池正在运行的线程”中取运行。

AbortPolicy(jdk默认策略，队列满并线程满时直接拒绝添加新任务，并抛出异常)

DiscardPolicy 抛弃当前任务

DiscardOldestPolicy 抛弃旧任务

调度策略

1. 请求到来首先交给coreSize内的常驻线程执行,如果poolsize小于coresize，那么只要来了一个request，就新创建一个thread来执行
2. 如果coreSize的线程全忙，也就是poolsize已经等于coresize，任务被放到队列里面
3. 如果队列放满了，会新增线程，直到达到maxSize
4. 如果还是处理不过来，会把一个异常扔到RejectedExecutionHandler中去，用户可以自己设定这种情况下的最终处理策略

对于大于coreSize而小于maxSize的那些线程，空闲了keepAliveTime后，会被销毁。观察上面说的优先级顺序可以看到，假如说给ExecutorService一个无限长的队列，比如LinkedBlockingQueue，那么maxSize>coreSize就是没有意义的 。

线程的复用

线程池中的线程,创建之后,并没有结束,而是一直循环从队列里取任务.取到runable对象就调用其run()方法!每个work线程,里面的run方法都是一个while循环,while循环条件里面有个阻塞方法!所谓的firstTask,就是线程刚创建的时候执行的第一个runable对象!一旦创建就阻塞的去队列里面去取!这里需要注意阻塞队列的选择,和大小!

线程池中线程发生异常的处理
线程池中的线程会捕获任何可能的Throwable异常,并且调用afterExecute方法,传递出去。
会调用 addWorker(null, false);方法,重新创建一个线程做为补充!
然后将异常抛出去, 如果是用execute()方法可以打印到控制台,如果是submit()提交,则会吞没掉,大概是因为future要拿到句柄吧,看下submit提交的代码
在AbstractExecutorService类里面
    /**
     * @throws RejectedExecutionException {@inheritDoc}
     * @throws NullPointerException       {@inheritDoc}
     */
    public Future<?> submit(Runnable task) {
        if (task == null) throw new NullPointerException();
        RunnableFuture<Void> ftask = newTaskFor(task, null);
        execute(ftask);
        return ftask;
    }
可以看到其实还是调用了execute方法,只不过多包装了一层,包装之后变成了callable对象,那么当我们调用task.run（）的时候,就是调用的是RunnableFuture的run方法,我们看他这个方法
  protected <T> RunnableFuture<T> newTaskFor(Runnable runnable, T value) {
        return new FutureTask<T>(runnable, value);
    }
我们看下FutureTask的run方法
public void run() {
        if (state != NEW ||
            !UNSAFE.compareAndSwapObject(this, runnerOffset,
                                         null, Thread.currentThread()))
            return;
        try {
            Callable<V> c = callable;
            if (c != null && state == NEW) {
                V result;
                boolean ran;
                try {
                    result = c.call();
                    ran = true;
//捕获了所有的异常,并且不会抛出
                } catch (Throwable ex) {
                    result = null;
                    ran = false;
                    setException(ex);
                }
                if (ran)
                    set(result);
            }
        } finally {
            // runner must be non-null until state is settled to
            // prevent concurrent calls to run()
            runner = null;
            // state must be re-read after nulling runner to prevent
            // leaked interrupts
            int s = state;
            if (s >= INTERRUPTING)
                handlePossibleCancellationInterrupt(s);
        }

.