Java线程状态，thread dump及分析

本文内容

1. 线程状态转换的介绍
2. 如何做 thread dump
3. 如何分析 thread dump
4. 如何使用 LockSupport 替换 suspend 和 resume

为何需要Thread dump

当应用运行效率与我们的预想不符时，需要查看应用内部是否存在死锁，I/O等待，锁等待，条件等待，线程竞争等，用以排除错误。

线程状态机：

图片来源: Core Java Vol 1, 9th Edition, Horstmann, Cay S. & Cornell, Gary_2013

• New：所有新建且还没有开始运行的线程，都处于此状态，意味着其代码还没开始执行，也没有开始调度。
• Runnable：准备好运行的线程都处于此状态，这些线程在任意时刻可能处于等待运行或者是运行中，具体取决于线程调度器。线程调度器会为每个线程分配固定的运行时间，线程运行一段时间后将被暂停和交出CPU，让其他处于等待运行的线程有机会来运行。
• Blocked/Waiting：线程暂时不活动时将进入此状态之一，例如等待I/O，访问已被加锁的临界区等。处于此状态的线程将不会继续运行且不消耗CPU，直到条件满足后，调度器重新把它的状态改为Runnable。
• Timed Waiting：线程调用了sleep或者wait(time)的方法后，将进入此状态，直到超时或者受到通知。
• Terminated：线程正常退出，遇到不寻常的错误或者没有处理的异常，都会进入此状态。处于此状态的线程不会再消耗CPU。

---

线程各种状态的例子

New State

package demo.thread.state;

public class NewState {
    public static void main(String[] args) {
        Thread t = new Thread();
        System.out.println("Thread state: " + t.getState());
    }
}

输出：

Thread state: NEW

Runnable State

package demo.thread.state;

import java.util.ArrayList;
import java.util.List;

import static java.lang.Thread.yield;

public class RunnableState {
    public static void main(String[] args) throws InterruptedException {
        List<Thread> threads = new ArrayList<>();
        int count = Runtime.getRuntime().availableProcessors() * 2;
        for (int i = 0; i < count; ++i) {
            Thread thread = new Thread(() -> {
                for (long j = 0; j < 10000000000000L; ++j) {
                    if (j % 1000 == 0)
                        yield();
                }
            });
            thread.setDaemon(true);
            threads.add(thread);
        }
        threads.forEach(Thread::start);
        for (int i = 0; i < 3; ++i) {
            if (i > 0)
                System.err.println("=================");
            Thread.sleep(1000);
            threads.forEach(thread -> System.err.println(thread.getName() + ": " + thread.getState()));
        }
    }
}

输出：

Thread-0: RUNNABLE
Thread-1: RUNNABLE
Thread-2: RUNNABLE
Thread-3: RUNNABLE
Thread-4: RUNNABLE
Thread-5: RUNNABLE
Thread-6: RUNNABLE
Thread-7: RUNNABLE
=================
Thread-0: RUNNABLE
Thread-1: RUNNABLE
Thread-2: RUNNABLE
Thread-3: RUNNABLE
Thread-4: RUNNABLE
Thread-5: RUNNABLE
Thread-6: RUNNABLE
Thread-7: RUNNABLE
=================
Thread-0: RUNNABLE
Thread-1: RUNNABLE
Thread-2: RUNNABLE
Thread-3: RUNNABLE
Thread-4: RUNNABLE
Thread-5: RUNNABLE
Thread-6: RUNNABLE
Thread-7: RUNNABLE

Blocked State

package demo.thread.state;

public class BlockedState {
    private static final Object lock = new Object();

    public static void main(String[] args) throws InterruptedException {
        Thread thread = new Thread(() -> {
            String threadName = Thread.currentThread().getName();
            System.err.println(threadName + ": Start to acquire lock...");
            synchronized (lock) {
                System.err.println(threadName + ": " + Thread.currentThread().getState());
                System.err.println(threadName + ": Lock acquired.");
            }
        });
        thread.setDaemon(true);

        synchronized (lock) {
            System.err.println(Thread.currentThread().getName() + ": Lock acquired.");
            thread.start();
            Thread.sleep(2000);
            System.err.println(thread.getName() + ": " + thread.getState());
            System.err.println(Thread.currentThread().getName() + ": Release Lock.");
        }
        thread.join();
    }
}

输出：

main: Lock acquired.
Thread-0: Start to acquire lock...
Thread-0: BLOCKED
main: Release Lock.
Thread-0: RUNNABLE
Thread-0: Lock acquired.

Waiting State

package demo.thread.state;

public class WaitingState {
    private static final Object lock = new Object();

    public static void main(String[] args) throws InterruptedException {
        Thread thread = new Thread(() -> {
            String threadName = Thread.currentThread().getName();
            System.err.println(threadName + ": Start to acquire lock...");
            synchronized (lock) {
                try {
                    System.err.println(threadName + ": Before lock wait.");
                    lock.wait();
                    System.err.println(threadName + ": After lock wait.");
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
        thread.setDaemon(true);

        thread.start();
        Thread.sleep(2000);
        System.err.println(thread.getName() + ": " + thread.getState());
        synchronized (lock) {
            System.err.println(Thread.currentThread().getName() + ": lock notify.");
            lock.notify();
        }
        Thread.sleep(1000);
        System.err.println(thread.getName() + ": " + thread.getState());
        thread.join();
    }
}

输出：

Thread-0: Start to acquire lock...
Thread-0: Before lock wait.
Thread-0: WAITING
main: lock notify.
Thread-0: After lock wait.
Thread-0: TERMINATED

Timed Waiting State

package demo.thread.state;

public class TimedWaitingState {
    private static final Object lock = new Object();

    public static void main(String[] args) throws InterruptedException {
        Thread thread = new Thread(() -> {
            String threadName = Thread.currentThread().getName();
            System.err.println(threadName + ": Start to acquire lock...");
            synchronized (lock) {
                try {
                    System.err.println(threadName + ": Before lock wait.");
                    lock.wait(1000);
                    System.err.println(threadName + ": After lock wait timeout.");
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
        thread.setDaemon(true);

        thread.start();
        Thread.sleep(500);
        System.err.println(thread.getName() + ": " + thread.getState());
        Thread.sleep(1000);
        System.err.println(thread.getName() + ": " + thread.getState());
        thread.join();
    }
}

输出：

Thread-0: Start to acquire lock...
Thread-0: Before lock wait.
Thread-0: TIMED_WAITING
Thread-0: After lock wait timeout.
Thread-0: TERMINATED

Terminated State

package demo.thread.state;

public class TerminatedState {
    public static void main(String[] args) throws InterruptedException {
        Thread thread = new Thread(() -> {
            System.err.println(Thread.currentThread().getName() + " is running.");
        });
        thread.start();
        thread.join();
        System.err.println(thread.getName() + ": " + thread.getState());
    }
}

输出：

Thread-0 is running.
Thread-0: TERMINATED

---

获取Thread Dump

查找java进程PID

命令1：jps -l
或者
命令2：ps -ef | grep java
注意：需要当前用户有足够权限
通过 “man jps” 可以查看完整的命令参数

获取Thread dump

命令：jstack -l
通过 “man jstack” 可以查看完整的命令参数

获取进程内所有线程的信息

命令：top -H -b -n1 -p $pid

• -H 表示top命令使用线程模式
• -b 表示用批处理
• -n 表示最大交互次数，因为我们指定了一个进程，所以设置为1
• -p 进程PID

---

推荐每隔5s获取一次thread dump，总共获取5~10次，然后进行分析。
以下为从eclipse收集来的脚本: jstackScript

#!/bin/bash
# Adaptation of script from eclipse.org http://wiki.eclipse.org/How_to_report_a_deadlock#jstackSeries_--_jstack_sampling_in_fixed_time_intervals_.28tested_on_Linux.29

if [ $# -eq 0 ]; then
    echo >&2 "Usage: jstackSeries <pid> [ <count> [ <delay> ] ]"
    echo >&2 "    Defaults: count = 10, delay = 1 (seconds)"
    exit 1
fi

pid=$1          # required
count=${2:-10}  # defaults to 10 times
delay=${3:-1} # defaults to 1 second

while [ $count -gt 0 ]
do
    jstack $pid >jstack.$pid.$(date +%s.%N)
    top -H -b -n1 -p $pid >top.$pid.$(date +%s.%N)
    sleep $delay
    let count--
    echo -n "."
done

命令：./jstackScript.sh $PID 10 5

---

Thread dump分析

生成 thread dump的脚本：run

#!/bin/bash

jps -l | grep $1 | awk '{print $1}' | xargs jstack -l > logs/$1.log

命令：./run $className

例子1：获取资源死锁

package demo.thread.dump;

public class DeadLock {
    public static void main(String[] args) throws InterruptedException {
        Object resourceA = new Object();
        Object resourceB = new Object();

        new Thread(() -> {
            synchronized (resourceA) {
                
                new Thread(() -> {
                    synchronized (resourceB) {
                        synchronized (resourceA) {
                        }
                    }
                }, "Thread-BBB").start();

                try {
                    Thread.sleep(1000);
                } catch (Exception e) {
                }
                synchronized (resourceB) {
                }
            }
        }, "Thread-AAA").start();
        Thread.sleep(10000000);
    }
}

运行：./run DeadLock
日志：

...
"Thread-BBB" #11 prio=5 os_prio=0 tid=0x00007fb750001800 nid=0x4d12 waiting for monitor entry [0x00007fb78ca0c000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at demo.thread.dump.DeadLock.lambda$null$0(DeadLock.java:14)
	- waiting to lock <0x00000000d68273e8> (a java.lang.Object)
	- locked <0x00000000d68273f8> (a java.lang.Object)
	at demo.thread.dump.DeadLock$$Lambda$2/1465806831.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None

"Thread-AAA" #10 prio=5 os_prio=0 tid=0x00007fb7a4259800 nid=0x4d11 waiting for monitor entry [0x00007fb78cb0d000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at demo.thread.dump.DeadLock.lambda$main$1(DeadLock.java:23)
	- waiting to lock <0x00000000d68273f8> (a java.lang.Object)
	- locked <0x00000000d68273e8> (a java.lang.Object)
	at demo.thread.dump.DeadLock$$Lambda$1/1078694789.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None
...

Found one Java-level deadlock:
=============================
"Thread-BBB":
  waiting to lock monitor 0x00007fb744002178 (object 0x00000000d68273e8, a java.lang.Object),
  which is held by "Thread-AAA"
"Thread-AAA":
  waiting to lock monitor 0x00007fb744003568 (object 0x00000000d68273f8, a java.lang.Object),
  which is held by "Thread-BBB"
  
Java stack information for the threads listed above:
===================================================
"Thread-BBB":
	at demo.thread.dump.DeadLock.lambda$null$0(DeadLock.java:14)
	- waiting to lock <0x00000000d68273e8> (a java.lang.Object)
	- locked <0x00000000d68273f8> (a java.lang.Object)
	at demo.thread.dump.DeadLock$$Lambda$2/1465806831.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)
"Thread-AAA":
	at demo.thread.dump.DeadLock.lambda$main$1(DeadLock.java:23)
	- waiting to lock <0x00000000d68273f8> (a java.lang.Object)
	- locked <0x00000000d68273e8> (a java.lang.Object)
	at demo.thread.dump.DeadLock$$Lambda$1/1078694789.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

Found 1 deadlock.

JVM帮我们检测到了死锁：

1. 线程BBB已经锁定了 0x00000000d68273f8 (resourceB)， 等待锁定 0x00000000d68273e8 (resourceA)
2. 线程AAA已经锁定了 0x00000000d68273e8 (resourceA)，等待锁定 0x00000000d68273f8 (resourceB)
3. 线程AAA和BBB的状态都是BLOCKED

例子2：ReentrantLock和共享对象的死锁

package demo.thread.dump;

import java.util.concurrent.locks.ReentrantReadWriteLock;

public class ReentrantDeadLockWithShareObject {
    private static final Object sharedObject = new Object();
    private static final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();

    public static void main(String[] args) throws InterruptedException {
        new Thread(() -> {
            lock.readLock().lock();

            new Thread(() -> {
                synchronized (sharedObject) {
                    lock.writeLock().lock();
                }
            }, "Thread-BBB").start();

            try {
                Thread.sleep(1000);
            } catch (Exception e) {
            }
            synchronized (sharedObject) {
            }
        }, "Thread-AAA").start();

        Thread.sleep(100000000);
    }
}

日志：

...
"Thread-BBB" #11 prio=5 os_prio=0 tid=0x00007fdf44001800 nid=0x5474 waiting on condition [0x00007fdf8145e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000d6831ea0> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
	at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
	at demo.thread.dump.ReentrantDeadLockWithShareObject.lambda$null$0(ReentrantDeadLockWithShareObject.java:15)
	- locked <0x00000000d6828420> (a java.lang.Object)
	at demo.thread.dump.ReentrantDeadLockWithShareObject$$Lambda$2/1584062783.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None

"Thread-AAA" #10 prio=5 os_prio=0 tid=0x00007fdf98232000 nid=0x5473 waiting for monitor entry [0x00007fdf8155f000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at demo.thread.dump.ReentrantDeadLockWithShareObject.lambda$main$1(ReentrantDeadLockWithShareObject.java:24)
	- waiting to lock <0x00000000d6828420> (a java.lang.Object)
	at demo.thread.dump.ReentrantDeadLockWithShareObject$$Lambda$1/558638686.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None
...

这次没那么走运，JVM没帮我们检测出死锁。从日志可以看出：

1. 线程BBB已经锁定了 0x00000000d6828420（sharedObject），等待锁定 0x00000000d6831ea0
2. 线程AAA等待锁定 0x00000000d6828420 (sharedObject)
3. 线程BBB的状态是 WAITING
4. 线程AAA的状态是 BLOCKED

例子3：繁忙的线程

package demo.thread.dump;

public class BusyRunning {
    public static void main(String[] args) throws InterruptedException {
        Thread busyThread = new Thread(() -> {
           for (long i = 0; i < 100000000000000L; ++i) {
           }
        }, "Thread-BUSY");
        busyThread.start();

        Thread.sleep(10000000);
    }
}

日志里面没有什么特别的东西。
使用脚本来获取 top的输出：run_top

#!/bin/bash

jps -l | grep $1 | awk '{print $1}' | xargs top -H -b -n1 -p > logs/$1.top

命令：./run_top BusyRunning
查看BusyRunning.top

...
  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
22079 helowken  20   0 4680700  39560  16956 R 93.8  0.5   4:39.21 java
22052 helowken  20   0 4680700  39560  16956 S  0.0  0.5   0:00.00 java
22055 helowken  20   0 4680700  39560  16956 S  0.0  0.5   0:00.25 java
22056 helowken  20   0 4680700  39560  16956 S  0.0  0.5   0:00.00 java
...

发现 PID=22079的线程CPU耗时最高，把22079转成16进制：
22079 / 16 = 1379 余 15 (F)
1379 / 16 = 86 余 3
86 / 16 = 5 余 6
5 / 16 = 0 余 5
所以22079的16进制就是 563F

命令：grep -i “563F” logs/BusyRunning.log -A10

"Thread-BUSY" #10 prio=5 os_prio=0 tid=0x00007fd56427f800 nid=0x563f runnable [0x00007fd54284c000]
   java.lang.Thread.State: RUNNABLE
	at demo.thread.dump.BusyRunning.lambda$main$0(BusyRunning.java:6)
	at demo.thread.dump.BusyRunning$$Lambda$1/1831932724.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None
...

能看出繁忙线程正在运行的代码：
at demo.thread.dump.BusyRunning.lambda$main$0(BusyRunning.java:6)

例子4：无限等待的I/O

package demo.thread.dump;

public class WaitingForIO {
    public static void main(String[] args) throws InterruptedException {
        Thread ioThread = new Thread(() -> {
            try {
                while (System.in.read() != -1) {
                }
            } catch (Exception e) {
            }
        }, "IORunning");
        ioThread.start();
        ioThread.join();
    }
}

日志：

"IORunning" #10 prio=5 os_prio=0 tid=0x00007f50002ae000 nid=0x5d82 runnable [0x00007f4fddd49000]
   java.lang.Thread.State: RUNNABLE
	at java.io.FileInputStream.readBytes(Native Method)
	at java.io.FileInputStream.read(FileInputStream.java:255)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
	- locked <0x00000000d671bed0> (a java.io.BufferedInputStream)
	at demo.thread.dump.WaitingForIO.lambda$main$0(WaitingForIO.java:7)
	at demo.thread.dump.WaitingForIO$$Lambda$1/1831932724.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
	- None

不管等多久，thread dump多少次，IO线程的状态永远是Runnable，但是永远结束不了。

---

Java LockSupport

最后看一下Java提供的 LockSupport，可以替代已经废弃的suspend和resume方法，但绝大多数时候，都不会直接使用。

LockSupport提供的park方法，用于暂停当前线程的调度；unpark方法用于给暂停的线程颁发一个允许，使其可以恢复运行。但是处于暂停的线程也可以被中断，所以在中断后需要判断条件是否已经满足，如果不满足，需要重新调用park方法来暂停。

例子1：park和unpark的使用

package demo.thread.lockSupport;

import java.util.concurrent.locks.LockSupport;

public class LockSupportSimpleTest {
    private static final Object lock = new Object();

    public static void main(String[] args) throws InterruptedException {
        Thread t = new Thread(() -> {
            String threadName = Thread.currentThread().getName();
            System.out.println("Park AAA");
            LockSupport.park(lock);
            System.out.println(threadName + " exits.");
        }, "AAA");
        t.start();

        Thread t2 = new Thread(() -> {
            String threadName = Thread.currentThread().getName();
            System.out.println("Park BBB");
            LockSupport.park(lock);
            System.out.println(threadName + " exits.");
        }, "BBB");
        t2.start();

        Thread.sleep(1000);
        System.out.println("Blocker of AAA equals to lock: " + (LockSupport.getBlocker(t) == lock));
        System.out.println("Blocker of BBB equals to lock: " + (LockSupport.getBlocker(t2) == lock));

        System.out.println("AAA status: " + t.getState());
        System.out.println("BBB status: " + t2.getState());

        System.out.println("unpark AAA");
        LockSupport.unpark(t);
        System.out.println("unpark BBB");
        LockSupport.unpark(t2);

        t.join();
        t2.join();
    }
}

输出：

Park AAA
Park BBB
Blocker of AAA equals to lock: true
Blocker of BBB equals to lock: true
AAA status: WAITING
BBB status: WAITING
unpark AAA
unpark BBB
AAA exits.
BBB exits.

例子2：有条件的等待

以下取自Java Doc的例子

package demo.thread.lockSupport;

import java.util.Queue;
import java.util.Random;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.LockSupport;

public class LockSupportConditionTest {
    private static AtomicBoolean locked = new AtomicBoolean(false);
    private static Queue<Thread> waiters = new ConcurrentLinkedQueue<Thread>();

    private static void lock() {
        boolean wasInterrupted = false;
        Thread current = Thread.currentThread();
        waiters.add(current);

        // Block while not first in queue or cannot acquire lock
        while (waiters.peek() != current ||
                !locked.compareAndSet(false, true)) {
            LockSupport.park();
            if (Thread.interrupted()) // ignore interrupts while waiting
                wasInterrupted = true;
        }

        waiters.remove();
        if (wasInterrupted)          // reassert interrupt status on exit
            current.interrupt();
    }

    private static void unlock() {
        locked.set(false);
        LockSupport.unpark(waiters.peek());
    }

    public static void main(String[] args) throws InterruptedException {
        AtomicInteger count = new AtomicInteger(5);
        CountDownLatch startLatch = new CountDownLatch(count.get());
        for (int i = 0, len = count.get(); i < len; ++i) {
            Thread t = new Thread(() -> {
                try {
                    Thread.sleep(new Random().nextInt(1000));
                } catch (InterruptedException e) {
                }
                startLatch.countDown();
                System.err.println(Thread.currentThread().getName() + " try to lock...");
                lock();
                System.err.println(Thread.currentThread().getName() + " locked successfully.");
                count.decrementAndGet();
            }, "TTT-" + i);
            t.start();
        }

        startLatch.await();
        System.err.println("Start to unlock...");
        while (count.get() > 0) {
            unlock();
        }
        System.err.println("Exits.");
    }
}

输出：

TTT-4 try to lock...
TTT-4 locked successfully.
TTT-2 try to lock...
TTT-0 try to lock...
TTT-1 try to lock...
TTT-3 try to lock...
Start to unlock...
TTT-2 locked successfully.
TTT-0 locked successfully.
TTT-1 locked successfully.
TTT-3 locked successfully.
Exits.

参考文章

• Lifecycle and States of a Thread in Java
• Take thread dumps from a JVM
• How to Analyze Java Thread Dumps