本文详细介绍了Java中实现阻塞队列BlockingQueue的原理和方法，包括ArrayBlockingQueue、LinkedBlockingQueue、PriorityBlockingQueue等，强调了阻塞队列在多线程环境中的便利性和在消费者-生产者模型中的应用。同时，文章还讨论了阻塞队列与非阻塞队列的区别，并提供了相关方法的对比。
java实现BlockingQueue接口的队列解析
java实现BlockingQueue接口的队列解析一jdk中的阻塞队列概况二阻塞队列与非阻塞队列中的方法对比
非阻塞队列常用方法阻塞队列常用方法 三各阻塞队列实现原理
ArrayBlockingQueueLinkerBlockingQueuePriorityBlockingQueueDelayQueueSynchronousQueueDelayedWorkQueueTransferQueue
阻塞队列与非阻塞队列一个最大的区别就是：阻塞队列能够阻塞当前试图从队列中获取元素的线程，而非阻塞队列不会。因此在面对类似消费者-生产者的模型时，使用非阻塞队列就必须额外地实现同步策略以及线程间唤醒策略，这个实现起来就非常麻烦。但是有了阻塞队列就不一样了，它会对当前线程产生阻塞，比如一个线程从一个空的阻塞队列中取元素，此时线程会被阻塞直到阻塞队列中有了元素。当队列中有元素后，被阻塞的线程会自动被唤醒(不需要我们编写代码去唤醒)。这样提供了极大的方便性。
一.jdk中的阻塞队列概况
阻塞队列由BlockingQueue进行定义。在jdk 1.8中实现了该接口的主要有以下几个： ArrayBlockingQueue –基于数组实现的一个阻塞队列，在创建ArrayBlockingQueue对象时必须制定容量大小。并且可以指定公平性与非公平性，默认情况下为非公平的，即不保证等待时间最长的线程最优先能够访问队列。 LinkedBlockingQueue–基于链表实现的一个阻塞队列，在创建LinkedBlockingQueue对象时如果不指定容量大小，则默认大小为Integer.MAX_VALUE。 PriorityBlockingQueue–***阻塞队列，它会按照元素的优先级对元素进行排序，按照优先级顺序出队，每次出队的元素都是优先级最高的元素。 DelayQueue –基于PriorityQueue实现的延迟队列，是一个***的阻塞队列，用于放置实现了Delayed接口的对象，其中的对象只能在其到期时才能从队列中取走。因此向队列中插入时永远不会阻塞，获取时才有可能被阻塞。 SynchronousQueue –同步阻塞队列，队列大小为1，一个元素要放到该队列中必须有一个线程在等待获取元素。 DelayedWorkQueue –该队列为ScheduledThreadPoolExecutor中的静态内部类，ScheduledThreadPoolExecutor便是通过该队列使得队列中的元素按一定顺序排列从而时延迟任务和周期性任务得以顺利执行。 BlockingDeque–双向阻塞队列的接口。 TransferQueue–接口，定义了另一种阻塞情况：生产者会一直阻塞直到所添加到队列的元素被某一个消费者所消费，而BlockingQueue只需将元素添加到队列中后生产者便会停止被阻塞。
二.阻塞队列与非阻塞队列中的方法对比
非阻塞队列常用方法
在非阻塞队列中常用的操作队列的方法主要是下面几种： add(E e):将元素e插入到队列末尾，如果插入成功，则返回true；如果插入失败(即队列已满)，则会抛出异常；
remove()：移除队首元素，若移除成功，则返回true；如果移除失败(队列为空)，则会抛出异常；
offer(E e)：将元素e插入到队列末尾，如果插入成功，则返回true；如果插入失败(即队列已满)，则返回false；
poll()：移除并获取队首元素，若成功，则返回队首元素；否则返回null；
peek()：获取队首元素，但不移除。若成功，则返回队首元素；否则返回null
对于非阻塞队列，一般情况下建议使用offer、poll和peek三个方法，不建议使用add和remove方法。原因看上面的描述很明显了：使用offer、poll和peek三个方法可以通过返回值判断操作成功与否，而使用add和remove方法需要捕获异常才能判断操作是否成功。另外需要注意非阻塞队列的这些方法都没有进行同步处理。
阻塞队列常用方法
阻塞队列也实现了Queue，因此也具有上述方法并且都进行了同步处理。除此之外还有4个很有用的方法： put(E e)：向队尾存入元素，如果队列满，则等待；
take()：从队首取元素，如果队列为空，则等待；
offer(E e,long timeout, TimeUnit unit)：向队尾存入元素，如果队列满，则等待一定的时间，当时间期限达到时，如果还没有插入成功，则返回false；否则返回true；
poll(long timeout, TimeUnit unit)：从队首取元素，如果队列空，则等待一定的时间，当时间期限达到时，如果取不到，则返回null；否则返回取得的元素；
三.各阻塞队列实现原理
ArrayBlockingQueue
首先看一下类中的成员变量
public class ArrayBlockingQueue extends AbstractQueue
implements BlockingQueue, java.io.Serializable {
/**
* Serialization ID. This class relies on default serialization
* even for the items array, which is default-serialized, even if
* it is empty. Otherwise it could not be declared final, which is
* necessary here.
*/
private static final long serialVersionUID = -817911632652898426L;
/** The queued items */
final Object[] items;
/** items index for next take, poll, peek or remove */
int takeIndex;
/** items index for next put, offer, or add */
int putIndex;
/** Number of elements in the queue */
int count;
/*
* Concurrency control uses the classic two-condition algorithm
* found in any textbook.
*/
/** Main lock guarding all access */
final ReentrantLock lock;
/** Condition for waiting takes */
private final Condition notEmpty;
/** Condition for waiting puts */
private final Condition notFull;
｝
可以看到使用数组来保存元素。takeIndex是队首下标，putIndex是队尾下标，count是队列中元素数目。lock是一个可重入锁，notEmpty和notFull是等待条件。关于可重入锁和Condition的相关知识将会在其他博客中阐述。 再来看看它的构造函数
//指定大小的队列
public ArrayBlockingQueue(int capacity) {
}
//指定大小和公平性的队列
public ArrayBlockingQueue(int capacity, boolean fair) {
}
//指定大小和公平性并使用已有的集合构建
public ArrayBlockingQueue(int capacity, boolean fair,
Collection extends E> c) {
}
在来看看它的两个关键方法put和take的实现方式。 首先看看put
public void put(E e) throws InterruptedException {
checkNotNull(e);
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
while (count == items.length)
notFull.await();
enqueue(e);
} finally {
lock.unlock();
}
}
private static void checkNotNull(Object v) {
if (v == null)
throw new NullPointerException();
}
/**
* Inserts element at current put position, advances, and signals.
* Call only when holding lock.
*/
private void enqueue(E x) {
// assert lock.getHoldCount() == 1;
// assert items[putIndex] == null;
final Object[] items = this.items;
items[putIndex] = x;
if (++putIndex == items.length)
putIndex = 0;
count++;
notEmpty.signal();
}
代码摘自jdk1.8，之前的代码和它稍有不同但核心没变。首先获取锁，判断当前队列是否满，满则阻塞，被其他线程唤醒时拆入元素，插入成功后通过notEmpty.signal()唤醒因队列为空等待取元素的线程。最后释放锁。 再来看看take
public E take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
while (count == 0)
notEmpty.await();
return dequeue();
} finally {
lock.unlock();
}
}
/**
* Extracts element at current take position, advances, and signals.
* Call only when holding lock.
*/
private E dequeue() {
// assert lock.getHoldCount() == 1;
// assert items[takeIndex] != null;
final Object[] items = this.items;
@SuppressWarnings("unchecked")
E x = (E) items[takeIndex];
items[takeIndex] = null;
if (++takeIndex == items.length)
takeIndex = 0;
count--;
if (itrs != null)
itrs.elementDequeued();
notFull.signal();
return x;
}
依然是先获取锁，若队列为空则等待，被其他线程唤醒后调用dequeue方法获取元素，获取成功后通过notFull.signal();唤醒因队列满无法放元素的线程
LinkerBlockingQueue
LinkerBlockingQueue使用node结构来存储数据
static class Node {
E item;
/**
* One of:
* - the real successor Node
* - this Node, meaning the successor is head.next
* - null, meaning there is no successor (this is the last node)
*/
Node next;
Node(E x) { item = x; }
}
put和take方法和ArrayBlockingQueue类似，这里就不阐述了，感兴趣的读者可以自行阅读源码
PriorityBlockingQueue
该队列为***队列并且会按照元素的优先级对元素进行排序。首先看一下成员变量
/**
* Default array capacity.
*/
private static final int DEFAULT_INITIAL_CAPACITY = 11;
/**
* The maximum size of array to allocate.
* Some VMs reserve some header words in an array.
* Attempts to allocate larger arrays may result in
* OutOfMemoryError: Requested array size exceeds VM limit
*/
private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8;
/**
* Priority queue represented as a balanced binary heap: the two
* children of queue[n] are queue[2*n+1] and queue[2*(n+1)]. The
* priority queue is ordered by comparator, or by the elements'
* natural ordering, if comparator is null: For each node n in the
* heap and each descendant d of n, n <= d. The element with the
* lowest value is in queue[0], assuming the queue is nonempty.
*/
private transient Object[] queue;
/**
* The number of elements in the priority queue.
*/
private transient int size;
/**
* The comparator, or null if priority queue uses elements'
* natural ordering.
*/
private transient Comparator super E> comparator;
/**
* Lock used for all public operations
*/
private final ReentrantLock lock;
/**
* Condition for blocking when empty
*/
private final Condition notEmpty;
/**
* Spinlock for allocation, acquired via CAS.
*/
private transient volatile int allocationSpinLock;
/**
* A plain PriorityQueue used only for serialization,
* to maintain compatibility with previous versions
* of this class. Non-null only during serialization/deserialization.
*/
private PriorityQueue q;
PriorityBlockingQueue使用数组保存元素，在空间不够时会进行扩容。每个参数的含义上面的英文注释写的很清楚了，为阅读方便这里再用中文简单解释一下： DEFAULT_INITIAL_CAPACITY － 初始时的数组大小 MAX_ARRAY_SIZE － 数组最大值 queue － 保存元素的数组，其实被实现为了一个最大堆 size － 保存队列中元素个数 comparator － 比较元素优先级使用的Comparator lock － 同步用的可重入锁 notEmpty － 同步用的Condition allocationSpinLock － 该int值配合unsafe类实现了一个同步用的自旋锁，后续代码中将能看到 q － 为了兼容老代码在序列化和反序列化时使用的队列，只有在序列化/反序列化时该队列中才有值。可以看到前面用于保存元素的queue被transient修饰，因此序列化时只能通过该对象q 在看看构造函数
public PriorityBlockingQueue() {
this(DEFAULT_INITIAL_CAPACITY, null);
}
public PriorityBlockingQueue(int initialCapacity) {
this(initialCapacity, null);
}
public PriorityBlockingQueue(int initialCapacity,
Comparator super E> comparator) {
...
}
public PriorityBlockingQueue(Collection extends E> c) {
...
}
提供4种构造函数，无参的构造函数初始队列大小为11，也可指定初始队列大小及比较方法，以及用已有的集合创建队列等。下面来看看核心的put和take方法。 首先看一下put
public void put(E e) {
offer(e); // never need to block
}
public boolean offer(E e) {
if (e == null)
throw new NullPointerException();
final ReentrantLock lock = this.lock;
lock.lock();
int n, cap;
Object[] array;
while ((n = size) >= (cap = (array = queue).length))
tryGrow(array, cap);
try {
Comparator super E> cmp = comparator;
if (cmp == null)
siftUpComparable(n, e, array);
else
siftUpUsingComparator(n, e, array, cmp);
size = n + 1;
notEmpty.signal();
} finally {
lock.unlock();
}
return true;
}
put委托给了offer实现，首先是获取锁，判断队列是否还能放入元素，不能则进行扩容，看看扩容方法的实现
private void tryGrow(Object[] array, int oldCap) {
lock.unlock(); // must release and then re-acquire main lock
Object[] newArray = null;
if (allocationSpinLock == 0 &&
UNSAFE.compareAndSwapInt(this, allocationSpinLockOffset,
0, 1)) {
try {
int newCap = oldCap + ((oldCap < 64) ?
(oldCap + 2) : // grow faster if small
(oldCap >> 1));
if (newCap - MAX_ARRAY_SIZE > 0) { // possible overflow
int minCap = oldCap + 1;
if (minCap < 0 || minCap > MAX_ARRAY_SIZE)
throw new OutOfMemoryError();
newCap = MAX_ARRAY_SIZE;
}
if (newCap > oldCap && queue == array)
newArray = new Object[newCap];
} finally {
allocationSpinLock = 0;
}
}
if (newArray == null) // back off if another thread is allocating
Thread.yield();
lock.lock();
if (newArray != null && queue == array) {
queue = newArray;
System.arraycopy(array, 0, newArray, 0, oldCap);
}
}
扩容时进行了同步处理。在数组小于64时每次扩容约2倍，大于64时每次扩容约1.5倍。 在回头继续看offer方法，扩容完成后如果传入comparator则调用siftUpComparable插入元素，看看其中的具体实现：
private static void siftUpComparable(int k, T x, Object[] array) {
Comparable super T> key = (Comparable super T>) x;
while (k > 0) {
int parent = (k - 1) >>> 1;
Object e = array[parent];
if (key.compareTo((T) e) >= 0)
break;
array[k] = e;
k = parent;
}
array[k] = key;
}
其实就是一个最大堆的插入过程，调整堆中节点位置使最大堆元素在堆顶即array［0］的位置。siftUpUsingComparator的实现与它基本一样，就不阐述了。 元素放入数组后调用notEmpty.signal();唤醒因队列空等待的线程并释放锁。 在看看take方法
public E take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
E result;
try {
while ( (result = dequeue()) == null)
notEmpty.await();
} finally {
lock.unlock();
}
return result;
}
若队列为空则等待，将获取元素委托给了dequeue()，看看它的实现
private E dequeue() {
int n = size - 1;
if (n < 0)
return null;
else {
Object[] array = queue;
E result = (E) array[0];
E x = (E) array[n];
array[n] = null;
Comparator super E> cmp = comparator;
if (cmp == null)
siftDownComparable(0, x, array, n);
else
siftDownUsingComparator(0, x, array, n, cmp);
size = n;
return result;
}
}
返回数组头部即堆顶的元素并对堆进行调整。看看调整堆的siftDownComparable方法
private static void siftDownComparable(int k, T x, Object[] array,
int n) {
if (n > 0) {
Comparable super T> key = (Comparable super T>)x;
int half = n >>> 1; // loop while a non-leaf
while (k < half) {
int child = (k << 1) + 1; // assume left child is least
Object c = array[child];
int right = child + 1;
if (right < n &&
((Comparable super T>) c).compareTo((T) array[right]) > 0)
c = array[child = right];
if (key.compareTo((T) c) <= 0)
break;
array[k] = c;
k = child;
}
array[k] = key;
}
}
一个典型的最大堆移除堆顶元素后的调整过程，siftDownUsingComparator与其类似，就不阐述了。 对最大堆的建立和调整还有疑问的同学可以参考这两篇博客： http://blog.csdn.net/kobejayandy/article/details/46832797 http://februus.iteye.com/blog/1288305
DelayQueue
还是先来看一下类中的成员变量
private final transient ReentrantLock lock = new ReentrantLock();
private final PriorityQueue q = new PriorityQueue();
private Thread leader = null;
private final Condition available = lock.newCondition();
可以看到，使用PriorityQueue来作为队列保存数据。其他几个变量含义都好理解，特殊的是这个leader，它是用来干嘛的呢？首先看一下jdk给出的解释:
/**
* Thread designated to wait for the element at the head of
* the queue. This variant of the Leader-Follower pattern
* (http://www.cs.wustl.edu/~schmidt/POSA/POSA2/) serves to
* minimize unnecessary timed waiting. When a thread becomes
* the leader, it waits only for the next delay to elapse, but
* other threads await indefinitely. The leader thread must
* signal some other thread before returning from take() or
* poll(...), unless some other thread becomes leader in the
* interim. Whenever the head of the queue is replaced with
* an element with an earlier expiration time, the leader
* field is invalidated by being reset to null, and some
* waiting thread, but not necessarily the current leader, is
* signalled. So waiting threads must be prepared to acquire
* and lose leadership while waiting.
*/
大致解释一下这段英文：这个leader变量用于存储等待队列中头元素的线程，使用这种领导者－追随者模式是为了减少多余的等待，领导者线程只用等待下一个元素能被获取，而其他线程则需继续等待成为领导者。领导者线程在进行take或者poll操作后需唤醒其他线程竞争领导者。当头元素发生变换时领导者或重置。 原谅我英语渣翻译得不好，总之简单一句话概括：这个Thread类型的变量即当前能获取到队列头元素的线程。稍后我们看源码能看到它是怎么被使用的。 再来看一下这个类的构造函数：
public DelayQueue() {}
public DelayQueue(Collection extends E> c) {
...
}
没有可多说的，一个无参构造函数和一个用指定集合构建队列的构造函数。 再看看核心的put和take方法。先看put
public void put(E e) {
offer(e);
}
public boolean offer(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
q.offer(e);
if (q.peek() == e) {
leader = null;
available.signal();
}
return true;
} finally {
lock.unlock();
}
}
依然是交给了offer实现，先加锁，然后放入元素，若放入的元素成为了头元素则将leader置为null并唤醒在等待available的线程。 再来看看take的实现
public E take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
for (;;) {
E first = q.peek();
if (first == null)
available.await();
else {
long delay = first.getDelay(NANOSECONDS);
if (delay <= 0)
return q.poll();
first = null; // don't retain ref while waiting
if (leader != null)
available.await();
else {
Thread thisThread = Thread.currentThread();
leader = thisThread;
try {
available.awaitNanos(delay);
} finally {
if (leader == thisThread)
leader = null;
}
}
}
}
} finally {
if (leader == null && q.peek() != null)
available.signal();
lock.unlock();
}
}
首先依然是加锁，然后获取队列中的头元素但是不移除，如果头元素为空则阻塞线程。若不为空，判断其等待时间是否结束，结束则返回。若等待时间未结束，判断当前是否已有leader线程，有则直接阻塞，无责将leader线程设置为当前线程，并设置其在到达delay时间前阻塞(即队列头元素的等待时间前)。 看到这里应该明白为什么当有新元素成为头元素时要设leader为null并且唤醒所有等待线程了吧？当有新元素成为头元素时原leader的等待时间已经无效了，此时所有等待的线程将再一次竞争leader并等待获取头元素。精妙的设计！
SynchronousQueue
这是一个很特殊的queue，它的内部没有任何存储空间。生产者线程对其的插入操作put必须等待消费者的移除操作take，反过来也一样。因此你不能调用peek()方法来看队列中是否有数据元素，因为数据元素只有当你试着取走的时候才可能存在，不取走而只想偷窥一下是不行的，当然遍历这个队列的操作也是不允许的。数据是在配对的生产者和消费者线程之间直接传递的，并不会将数据缓冲数据到队列中。可以这样来理解：生产者和消费者互相等待对方，握手，然后一起离开。 首先看一下构造函数签名
public SynchronousQueue() {
this(false);
}
public SynchronousQueue(boolean fair) {
transferer = fair ? new TransferQueue() : new TransferStack();
}
两种构造函数，无参的构造函数创建非公平的SynchronousQueue，或是可以指定SynchronousQueue的公平性。可以看到是否公平是看transferer的实力化对象来决定的。非公平竞争模式使用的数据结构TransferStack是后进先出栈(Lifo Stack)；公平竞争模式使用的数据结构TransferQueue是先进先出队列(Fifo Queue)，但从性能上来讲非公平模式更快。 再来看看put和take方法
public E take() throws InterruptedException {
E e = transferer.transfer(null, false, 0);
if (e != null)
return e;
Thread.interrupted();
throw new InterruptedException();
}
public void put(E e) throws InterruptedException {
if (e == null) throw new NullPointerException();
if (transferer.transfer(e, false, 0) == null) {
Thread.interrupted();
throw new InterruptedException();
}
}
可以看到都是调用了transferer.transfer方法来完成put和take。因此下面我们来看看SynchronousQueue中的核心：Transferer接口。
abstract static class Transferer {
abstract E transfer(E e, boolean timed, long nanos);
}
方法transfer中3个参数的含义分别为： e – 为空则是需要消费元素，不为空则是要放入的元素 timed – 是否允许超市 nanos – 超时时间，纳秒级别 在SynchronousQueue给出了该接口的两个实现：TransferQueue和TransferStack。TransferQueue用于构建公平的SynchronousQueue，TransferStack用于构建非公平的SynchronousQueue。首先我们来看看TransferQueue中transfer方法的实现：
E transfer(E e, boolean timed, long nanos) {
QNode s = null; // constructed/reused as needed
boolean isData = (e != null);
for (;;) {
QNode t = tail;
QNode h = head;
if (t == null || h == null) // saw uninitialized value
continue; // spin
if (h == t || t.isData == isData) { // empty or same-mode
QNode tn = t.next;
if (t != tail) // inconsistent read
continue;
if (tn != null) { // lagging tail
advanceTail(t, tn);
continue;
}
if (timed && nanos <= 0) // can't wait
return null;
if (s == null)
s = new QNode(e, isData);
if (!t.casNext(null, s)) // failed to link in
continue;
advanceTail(t, s); // swing tail and wait
Object x = awaitFulfill(s, e, timed, nanos);
if (x == s) { // wait was cancelled
clean(t, s);
return null;
}
if (!s.isOffList()) { // not already unlinked
advanceHead(t, s); // unlink if head
if (x != null) // and forget fields
s.item = s;
s.waiter = null;
}
return (x != null) ? (E)x : e;
} else { // complementary-mode
QNode m = h.next; // node to fulfill
if (t != tail || m == null || h != head)
continue; // inconsistent read
Object x = m.item;
if (isData == (x != null) || // m already fulfilled
x == m || // m cancelled
!m.casItem(x, e)) { // lost CAS
advanceHead(h, m); // dequeue and retry
continue;
}
advanceHead(h, m); // successfully fulfilled
LockSupport.unpark(m.waiter);
return (x != null) ? (E)x : e;
}
}
}
QNode是TransferQueue中的静态内部类，保存节点信息
static final class QNode {
volatile QNode next; // next node in queue
volatile Object item; // CAS'ed to or from null
volatile Thread waiter; // to control park/unpark
final boolean isData;
QNode(Object item, boolean isData) {
this.item = item;
this.isData = isData;
}
boolean casNext(QNode cmp, QNode val) {
return next == cmp &&
UNSAFE.compareAndSwapObject(this, nextOffset, cmp, val);
}
boolean casItem(Object cmp, Object val) {
return item == cmp &&
UNSAFE.compareAndSwapObject(this, itemOffset, cmp, val);
}
/**
* Tries to cancel by CAS'ing ref to this as item.
*/
void tryCancel(Object cmp) {
UNSAFE.compareAndSwapObject(this, itemOffset, cmp, this);
}
boolean isCancelled() {
return item == this;
}
/**
* Returns true if this node is known to be off the queue
* because its next pointer has been forgotten due to
* an advanceHead operation.
*/
boolean isOffList() {
return next == this;
}
// Unsafe mechanics
private static final sun.misc.Unsafe UNSAFE;
private static final long itemOffset;
private static final long nextOffset;
static {
try {
UNSAFE = sun.misc.Unsafe.getUnsafe();
Class> k = QNode.class;
itemOffset = UNSAFE.objectFieldOffset
(k.getDeclaredField("item"));
nextOffset = UNSAFE.objectFieldOffset
(k.getDeclaredField("next"));
} catch (Exception e) {
throw new Error(e);
}
}
}
总体分析，该方法中的同步策略都没有使用锁，同步通过spin(我理解为忙等)和cas(Compare and Swap)的方法来实现。 这段代码的实现逻辑我直接翻译jdk中的原文吧，原文如下：
/* Basic algorithm is to loop trying to take either of * two actions: * * 1. If queue apparently empty or holding same-mode nodes, * try to add node to queue of waiters, wait to be * fulfilled (or cancelled) and return matching item. * * 2. If queue apparently contains waiting items, and this * call is of complementary mode, try to fulfill by CAS’ing * item field of waiting node and dequeuing it, and then * returning matching item. * * In each case, along the way, check for and try to help * advance head and tail on behalf of other stalled/slow * threads. * * The loop starts off with a null check guarding against * seeing uninitialized head or tail values. This never * happens in current SynchronousQueue, but could if * callers held non-volatile/final ref to the * transferer. The check is here anyway because it places * null checks at top of loop, which is usually faster * than having them implicitly interspersed. */
大致谈一下自己的理解：队列中保存的都是执行相同操作的线程(都是take或都是put，即都是生产者或都是消费者)，当另一种类型的线程进入后，与队列中的头元素进行匹配，完成数据的传输。
再来看看TransferStack中transfer的实现
E transfer(E e, boolean timed, long nanos) {
SNode s = null; // constructed/reused as needed
int mode = (e == null) ? REQUEST : DATA;
for (;;) {
SNode h = head;
if (h == null || h.mode == mode) { // empty or same-mode
if (timed && nanos <= 0) { // can't wait
if (h != null && h.isCancelled())
casHead(h, h.next); // pop cancelled node
else
return null;
} else if (casHead(h, s = snode(s, e, h, mode))) {
SNode m = awaitFulfill(s, timed, nanos);
if (m == s) { // wait was cancelled
clean(s);
return null;
}
if ((h = head) != null && h.next == s)
casHead(h, s.next); // help s's fulfiller
return (E) ((mode == REQUEST) ? m.item : s.item);
}
} else if (!isFulfilling(h.mode)) { // try to fulfill
if (h.isCancelled()) // already cancelled
casHead(h, h.next); // pop and retry
else if (casHead(h, s=snode(s, e, h, FULFILLING|mode))) {
for (;;) { // loop until matched or waiters disappear
SNode m = s.next; // m is s's match
if (m == null) { // all waiters are gone
casHead(s, null); // pop fulfill node
s = null; // use new node next time
break; // restart main loop
}
SNode mn = m.next;
if (m.tryMatch(s)) {
casHead(s, mn); // pop both s and m
return (E) ((mode == REQUEST) ? m.item : s.item);
} else // lost match
s.casNext(m, mn); // help unlink
}
}
} else { // help a fulfiller
SNode m = h.next; // m is h's match
if (m == null) // waiter is gone
casHead(h, null); // pop fulfilling node
else {
SNode mn = m.next;
if (m.tryMatch(h)) // help match
casHead(h, mn); // pop both h and m
else // lost match
h.casNext(m, mn); // help unlink
}
}
}
}
看看jdk对这段代码的解释
/* * Basic algorithm is to loop trying one of three actions: * * 1. If apparently empty or already containing nodes of same * mode, try to push node on stack and wait for a match, * returning it, or null if cancelled. * * 2. If apparently containing node of complementary mode, * try to push a fulfilling node on to stack, match * with corresponding waiting node, pop both from * stack, and return matched item. The matching or * unlinking might not actually be necessary because of * other threads performing action 3: * * 3. If top of stack already holds another fulfilling node, * help it out by doing its match and/or pop * operations, and then continue. The code for helping * is essentially the same as for fulfilling, except * that it doesn’t return the item. */
循环等待3种行为中的一种发生： 1.如果栈为空或存在相同模式的节点(同为消费者或生产者)则将新节点加入栈中等待被匹配； 2.如果新节点与栈顶节点匹配，则将新节点入栈并与其匹配，匹配完成后弹出两个节点并返回值 3.如果新节点加入时发现另一个节点正在匹配，则帮助它匹配完成后再继续。 关于synchronousqueue还有不清楚的同学，强烈推荐这篇论文：http://www.cs.rochester.edu/u/scott/papers/2009_Scherer_CACM_SSQ.pdf 也可以参考这篇博文:http://blog.csdn.net/jy3161286/article/details/22993963?utm_source=tuicool&utm_medium=referral
DelayedWorkQueue
该队列作为静态内部类实现在ScheduledThreadPoolExecutor中。首先来看看它其中的变量
private static final int INITIAL_CAPACITY = 16;
private RunnableScheduledFuture>[] queue =
new RunnableScheduledFuture>[INITIAL_CAPACITY];
private final ReentrantLock lock = new ReentrantLock();
private int size = 0;
private Thread leader = null;
private final Condition available = lock.newCondition();
使用数组来存储队列中的数据，实际上数组被用来实现堆。初始化数组大小16。 再来看看put
public void put(Runnable e) {
offer(e);
}
public boolean offer(Runnable x) {
if (x == null)
throw new NullPointerException();
RunnableScheduledFuture> e = (RunnableScheduledFuture>)x;
final ReentrantLock lock = this.lock;
lock.lock();
try {
int i = size;
if (i >= queue.length)
grow();
size = i + 1;
if (i == 0) {
queue[0] = e;
setIndex(e, 0);
} else {
siftUp(i, e);
}
if (queue[0] == e) {
leader = null;
available.signal();
}
} finally {
lock.unlock();
}
return true;
}
有没有觉得很眼熟？和PriorityBlockingQueue的put大同小异。实际上扩容和放入元素并调整堆的方法也是类似的
private void grow() {
int oldCapacity = queue.length;
int newCapacity = oldCapacity + (oldCapacity >> 1); // grow 50%
if (newCapacity < 0) // overflow
newCapacity = Integer.MAX_VALUE;
queue = Arrays.copyOf(queue, newCapacity);
}
private void siftUp(int k, RunnableScheduledFuture> key) {
while (k > 0) {
int parent = (k - 1) >>> 1;
RunnableScheduledFuture> e = queue[parent];
if (key.compareTo(e) >= 0)
break;
queue[k] = e;
setIndex(e, k);
k = parent;
}
queue[k] = key;
setIndex(key, k);
}
就不阐述了，有疑问的同学可以翻到上面看看对PriorityBlockingQueue实现的解释。 在看看take
public RunnableScheduledFuture> take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
for (;;) {
RunnableScheduledFuture> first = queue[0];
if (first == null)
available.await();
else {
long delay = first.getDelay(NANOSECONDS);
if (delay <= 0)
return finishPoll(first);
first = null; // don't retain ref while waiting
if (leader != null)
available.await();
else {
Thread thisThread = Thread.currentThread();
leader = thisThread;
try {
available.awaitNanos(delay);
} finally {
if (leader == thisThread)
leader = null;
}
}
}
}
} finally {
if (leader == null && queue[0] != null)
available.signal();
lock.unlock();
}
}
和PriorityBlockingQueue一样，不详述了
TransferQueue
这是一个接口，新添加的transfer方法用来实现这种约束：生产者会一直阻塞直到所添加到队列的元素被某一个消费者所消费。顾名思义，阻塞就是发生在元素从一个线程transfer到另一个线程的过程中，它有效地实现了元素在线程之间的传递。除了transfer还包括一些其他方法，看看该接口的方法列表：
boolean tryTransfer(E e);//若当前存在一个正在等待获取的消费者线程(使用take()或者poll()函数)，使用该方法会即刻转移/传输对象元素e；若不存在，则返回false，并且不进入队列。这是一个不阻塞的操作。
void transfer(E e) throws InterruptedException;//若当前存在一个正在等待获取的消费者线程，即立刻移交之；否则，会插入当前元素e到队列尾部，并且等待进入阻塞状态，到有消费者线程取走该元素。
boolean tryTransfer(E e, long timeout, TimeUnit unit) throws InterruptedException;//若当前存在一个正在等待获取的消费者线程，会立即传输给它;否则将插入元素e到队列尾部，并且等待被消费者线程获取消费掉；若在指定的时间内元素e无法被消费者线程获取，则返回false，同时该元素被移除。(不加入队列)
boolean hasWaitingConsumer();//判断是否存在消费者线程
int getWaitingConsumerCount();//获取所有等待获取元素的消费线程数量。
关于TransferQueue存在的意义和性能可以操考下面的资料
当我第一次看到TransferQueue时，首先想到了已有的实现类SynchronousQueue。SynchronousQueue的队列长度为0，最初我认为这好像没多大用处，但后来我发现它是整个Java Collection Framework中最有用的队列实现类之一，特别是对于两个线程之间传递元素这种用例。
TransferQueue相比SynchronousQueue用处更广、更好用，因为你可以决定是使用BlockingQueue的方法(译者注：例如put方法)还是确保一次传递完成(译者注：即transfer方法)。在队列中已有元素的情况下，调用transfer方法，可以确保队列中被传递元素之前的所有元素都能被处理。Doug Lea说从功能角度来讲，LinkedTransferQueue实际上是ConcurrentLinkedQueue、SynchronousQueue(公平模式)和LinkedBlockingQueue的超集。而且LinkedTransferQueue更好用，因为它不仅仅综合了这几个类的功能，同时也提供了更高效的实现。
Joe Bowbeer提供了一篇William Scherer, Doug Lea, and Michael Scott的论文，在这篇论文中展示了LinkedTransferQueue的算法，性能测试的结果表明它优于Java 5的那些类(译者注：ConcurrentLinkedQueue、SynchronousQueue和LinkedBlockingQueue)。LinkedTransferQueue的性能分别是SynchronousQueue的3倍(非公平模式)和14倍(公平模式)。因为像ThreadPoolExecutor这样的类在任务传递时都是使用SynchronousQueue，所以使用LinkedTransferQueue来代替SynchronousQueue也会使得ThreadPoolExecutor得到相应的性能提升。考虑到executor在并发编程中的重要性，你就会理解添加这个实现类的重要性了。 引自http://ifeve.com/java-transfer-queue/
jdk1.8中唯一实现了该接口的实现时LinkedTransferQueue，下面我们从该类进行研究。 首先看一下LinkedTransferQueue中实现队列的数据结构node
static final class Node {
final boolean isData; //是不是数据，是的话item放具体东西
volatile Object item; // 如果不是数据则为null
volatile Node next;//指向下一个节点
volatile Thread waiter; // null until waiting
}
LinkedTransferQueue中无论是transfer、tryTransfer、put、take方法都是调用的xfer方法，因此直接来看该方法的实现了
/**
* Implements all queuing methods. See above for explanation.
*
* @param e the item or null for take
* @param haveData true if this is a put, else a take
* @param how NOW, ASYNC, SYNC, or TIMED
* @param nanos timeout in nanosecs, used only if mode is TIMED
* @return an item if matched, else e
* @throws NullPointerException if haveData mode but e is null
*/
private E xfer(E e, boolean haveData, int how, long nanos) {
if (haveData && (e == null))
throw new NullPointerException();
Node s = null; // the node to append, if needed
retry:
for (;;) { // restart on append race
for (Node h = head, p = h; p != null;) { // 如果头结点为空则跳过，非空进去找第一个可用节点
boolean isData = p.isData;
Object item = p.item;
if (item != p && (item != null) == isData) { // // 判断原节点可用性，如data的item应该是数值，如果是null则表明用过了
if (isData == haveData) // 两个节点是相同类型，无法match
break;
if (p.casItem(item, e)) { // 节点不同类型，match成功，更改原节点item，表明不可用
for (Node q = p; q != h;) {//更新头节点
Node n = q.next; // update by 2 unless singleton
if (head == h && casHead(h, n == null ? q : n)) {
h.forgetNext();
break;
} // advance and retry
if ((h = head) == null ||
(q = h.next) == null || !q.isMatched())
break; // unless slack < 2
}
LockSupport.unpark(p.waiter);
return LinkedTransferQueue.cast(item);
}
}
Node n = p.next;
p = (p != n) ? n : (h = head); // Use head if p offlist
}
if (how != NOW) { // 匹配失败，把新节点放进栈内，并根据参数决定立刻返回或者等待返回
if (s == null)
s = new Node(e, haveData);
Node pred = tryAppend(s, haveData);尝试添加新node
if (pred == null)
continue retry; // 不成功则重试整个过程
if (how != ASYNC)
return awaitMatch(s, pred, e, (how == TIMED), nanos);//根据参数，等不等别人放数据，拿数据，等多久
}
return e; // not waiting
}
}
代码注释的翻译参考自：http://www.cnblogs.com/rockman12352/p/3790245.html?utm_source=tuicool&utm_medium=referral 但博文中给出的示例有误，示例可参考下面： 给4个示例更方便理解： 情况一： 1：Head->Data Input->Data 队列中已有元素，再放入元素
Match: 根据他们的属性发现 cannot match ，因为是同类的
处理节点: 把新的data放在原来的data后面 HEAD->DATA->DATA
情况二： 2：Head->waiter Input->waiter 队列中已有消费者等待，再获取元素
Match: 根据他们的属性发现 cannot match ，因为是同类的
处理节点: 把新的waiter放在原来的waiter后面，然后head往后移一位。
HEAD->waiter->waiter
情况三： 3：Head->waiter Input->Data 队列中已有消费者等待，添加数据
Match: 成功match，就把waiter对应node的item变为Data的值(有主了)，并且叫waiter来取
处理节点：head后移一位
HEAD=waiter(用过)
情况四： 4：Head->Data Input->waiter 队列中已有数据，新加入消费者
Match: 成功match，就把Data的item变为waiter的值(即null,有主了)，并且返回数据。
处理节点：head后移一位
HEAD=DATA(用过)
看到一篇博文中说该类有bug，有兴趣的同学可以看看： http://ifeve.com/buglinkedtransferqueue-bug/ 经测试在目前最新的jdk “1.8.0_91”中博主所说的’bug’仍然存在。简单来说在这种情况下会发生’bug’： 队列为空或队列前面的元素已全部完成匹配，此时一个消费者线程进入，假设为t1，因队列空或已全部匹配，其无法匹配，准备开始运行tryAppend方法，
Node pred = tryAppend(s, haveData);
此时又进入一个与t1模式相同的线程抢占了cpu(即也是消费者线程)，设为t2。此时t1还未完成tryAppend方法的运行，所以队列仍然为空，t2准备开始运行tryAppend方法。此时又进入一个生产者线程t3抢占了cpu，因为t1,t2都没有完成tryAppend方法的运行，所以队列仍然为空，t3准备运行tryAppend方法。(以上步骤顺序可随意颠倒，即可以是t3先运行到tryAppend处阻塞，然后t2,t1抢占，等等。关键在3个线程都阻塞在了tryAppend开始处。)此时t1重新获得cpu，运行完成tryAppend方法将自己加入队列，t2重新获得cpu，运行完成tryAppend方法将自己加入队列。此时t2的中断被出发，t2中断，按LinkedTransferQueue的逻辑t1和t2变为匹配状态。t3重新获得cpu，执行tryAppend。按常理此时执行tryAppend应该失败，然后重新进入xfer()的循环中，因为队列中已有两个与t3相反模式的节点。但是因为LinkedTransferQueue判断t1和t2已匹配，队列即将为空，所以t3也成功加入了队列。 此时队列状况为： t1->t2->t3 假设接下来又进入一个消费者线程t4，在执行到xfer()的这里时将直接break
if (isData == haveData) // can't match
break;
代码中isData取值来自
boolean isData = p.isData;
p指向头节点。haveData即标明t4是消费者还是生产者。显然这里isData和haveData都是fales 接下来执行这段代码
if (how != NOW) { // No matches available
if (s == null)
s = new Node(e, haveData);
Node pred = tryAppend(s, haveData);
if (pred == null)
continue retry; // lost race vs opposite mode
if (how != ASYNC)
return awaitMatch(s, pred, e, (how == TIMED), nanos);
}
因为无法匹配t4也执行tryAppend尝试加入队列，在加入队列时会判断尾节点模式是否和t4一样，上面说了此时尾节点是t3，模式是生产者与t4相反，所以t4加入队列失败，返回null，即执行代码
if (pred == null)
continue retry;
又进入了一开始的循环：和头节点模式相同，判断无法完成匹配，尝试加入队列，加入队列时发现尾节点模式和自己相反，无法加入，重试，判断头节点模式相同，无法完成匹配…
t4就这样无限循环了 T T 此时如果有更多消费者线程进来，都将在这里无限循环，cpu飙升 但是！！ 如果这时有一个新的生成者线程t5进入，从xfer()方法的头部开始执行，判断头元素与自己模式相反，可以完成匹配，运行到这里
LockSupport.unpark(p.waiter);
刚才阻塞的t1线程将被唤醒，并且能够接受数据数据。头节点将重新指向t3(因为t2已中断，是无效节点)，t4的无限循环也被打破，和t5完成匹配，队列恢复正常。
so，在实际使用中不光这个bug出现概率极低，就算出现后只要一个新的生成者到来就能自己恢复正常。不过你可能还是会担心，万一这段时间到来的全是消费者cpu不是gg？为了避免这种情况，jdk1.8中在该类新增了扫描的机制，
* In addition to minimizing garbage retention via self-linking
* described above, we also unlink removed interior nodes. These
* may arise due to timed out or interrupted waits, or calls to
* remove(x) or Iterator.remove. Normally, given a node that was
* at one time known to be the predecessor of some node s that is
* to be removed, we can unsplice s by CASing the next field of
* its predecessor if it still points to s (otherwise s must
* already have been removed or is now offlist). But there are two
* situations in which we cannot guarantee to make node s
* unreachable in this way: (1) If s is the trailing node of list
* (i.e., with null next), then it is pinned as the target node
* for appends, so can only be removed later after other nodes are
* appended. (2) We cannot necessarily unlink s given a
* predecessor node that is matched (including the case of being
* cancelled): the predecessor may already be unspliced, in which
* case some previous reachable node may still point to s.
* (For further explanation see Herlihy & Shavit "The Art of
* Multiprocessor Programming" chapter 9). Although, in both
* cases, we can rule out the need for further action if either s
* or its predecessor are (or can be made to be) at, or fall off
* from, the head of list.
*
* Without taking these into account, it would be possible for an
* unbounded number of supposedly removed nodes to remain
* reachable. Situations leading to such buildup are uncommon but
* can occur in practice; for example when a series of short timed
* calls to poll repeatedly time out but never otherwise fall off
* the list because of an untimed call to take at the front of the
* queue.
*
* When these cases arise, rather than always retraversing the
* entire list to find an actual predecessor to unlink (which
* won't help for case (1) anyway), we record a conservative
* estimate of possible unsplice failures (in "sweepVotes").
* We trigger a full sweep when the estimate exceeds a threshold
* ("SWEEP_THRESHOLD") indicating the maximum number of estimated
* removal failures to tolerate before sweeping through, unlinking
* cancelled nodes that were not unlinked upon initial removal.
* We perform sweeps by the thread hitting threshold (rather than
* background threads or by spreading work to other threads)
* because in the main contexts in which removal occurs, the
* caller is already timed-out, cancelled, or performing a
* potentially O(n) operation (e.g. remove(x)), none of which are
* time-critical enough to warrant the overhead that alternatives
* would impose on other threads.
*
* Because the sweepVotes estimate is conservative, and because
* nodes become unlinked "naturally" as they fall off the head of
* the queue, and because we allow votes to accumulate even while
* sweeps are in progress, there are typically significantly fewer
* such nodes than estimated. Choice of a threshold value
* balances the likelihood of wasted effort and contention, versus
* providing a worst-case bound on retention of interior nodes in
* quiescent queues. The value defined below was chosen
* empirically to balance these under various timeout scenarios.
*
* Note that we cannot self-link unlinked interior nodes during
* sweeps. However, the associated garbage chains terminate when
* some successor ultimately falls off the head of the list and is
* self-linked.
*/
我们刚才遭遇的就是上面的case2了。这里也解释了为什么不使用http://ifeve.com/buglinkedtransferqueue-bug/ 这篇博文中的方法解决该问题。 所以在jdk1.8中，愉快的使用LinkedTransferQueue吧^ ^
扫一扫
1890
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
