并发编程笔记——第七章 Java并发包中并发队列原理剖析

最新推荐文章于 2022-03-10 04:42:06 发布

_大聪明

最新推荐文章于 2022-03-10 04:42:06 发布

阅读量128

点赞数

分类专栏：读书笔记文章标签：队列链表 java 多线程

本文链接：https://blog.csdn.net/weixin_42752999/article/details/110776174

版权

读书笔记专栏收录该内容

9 篇文章 0 订阅

订阅专栏

JDK中提供了一系列场景的并发安全队列。总体来说，按照实现方式的不同可分为阻塞队列和非阻塞队列，前者使用锁实现，后者则使用CAS非阻塞算法实现

一、ConcurrentLinkedQueue原理探究

线程安全的无界非阻塞队列，底层数据结构使用单向链表实现，入队出队操作使用CAS来实现线程安全

类图结构

在这里插入图片描述

原理介绍

offer操作：在队列末尾添加元素

public boolean offer(E e) {
    checkNotNull(e);
    // 构造Node节点，构造函数内部调用unsafe.putObject
    final Node<E> newNode = new Node<E>(e);
	// 从尾节点进行插入
    for (Node<E> t = tail, p = t;;) {
        Node<E> q = p.next;
        if (q == null) {
            // 使用CAS设置p节点的next节点
            if (p.casNext(null, newNode)) {
                if (p != t) // hop two nodes at a time
                    casTail(t, newNode);  // Failure is OK.
                return true;
            }
        }
        else if (p == q)
        	// 多线程操作时，由于poll操作移除元素后可能会把head变为自引用，也就是head的next变成了head，所以这里需要重新找新的head
            p = (t != (t = tail)) ? t : head;
        else
            // 寻找尾节点
            p = (p != t && t != (t = tail)) ? t : q;
    }
}

add()

public boolean add(E e) {
    return offer(e);
}

poll()：获取并移除队列头

public E poll() {
	// goto标记
    restartFromHead:
    for (;;) {
        for (Node<E> h = head, p = h, q;;) {
            E item = p.item;
			// 将p的item设置为null
            if (item != null && p.casItem(item, null)) {
                if (p != h) // hop two nodes at a time
                    updateHead(h, ((q = p.next) != null) ? q : p);
                return item;
            }
            else if ((q = p.next) == null) {
                updateHead(h, p);
                return null;
            }
            else if (p == q)
                continue restartFromHead;
            else
                p = q;
        }
    }
}

final void updateHead(Node<E> h, Node<E> p) {
    if (h != p && casHead(h, p))
    	// 将head设置为自引用
        h.lazySetNext(h);
}

peek()：读队列头元素

public E peek() {
    restartFromHead:
    for (;;) {
        for (Node<E> h = head, p = h, q;;) {
            E item = p.item;
            if (item != null || (q = p.next) == null) {
                updateHead(h, p);
                return item;
            }
            else if (p == q)
                continue restartFromHead;
            else
                p = q;
        }
    }
}

size()：计算队列元素个数。不精确，因为CAS不加锁，调用size期间可能增删元素

public int size() {
    int count = 0;
    for (Node<E> p = first(); p != null; p = succ(p))
        if (p.item != null)
            // 最大值为Integer.MAX_VALUE
            if (++count == Integer.MAX_VALUE)
                break;
    return count;
}

Node<E> first() {
    restartFromHead:
    for (;;) {
        for (Node<E> h = head, p = h, q;;) {
            boolean hasItem = (p.item != null);
            if (hasItem || (q = p.next) == null) {
                updateHead(h, p);
                return hasItem ? p : null;
            }
            else if (p == q)
                continue restartFromHead;
            else
                p = q;
        }
    }
}

final Node<E> succ(Node<E> p) {
    Node<E> next = p.next;
    return (p == next) ? head : next;
}

remove(Object o)：移除o，存在多个o则移除第一个

public boolean remove(Object o) {
    if (o != null) {
        Node<E> next, pred = null;
        for (Node<E> p = first(); p != null; pred = p, p = next) {
            boolean removed = false;
            E item = p.item;
            if (item != null) {
                if (!o.equals(item)) {
                    next = succ(p);
                    continue;
                }
                removed = p.casItem(item, null);
            }

            next = succ(p);
            if (pred != null && next != null) // unlink
                pred.casNext(p, next);
            if (removed)
                return true;
        }
    }
    return false;
}

contains()：是否包含。调用后其他线程添加无法感知，但调用后其他线程删除可以感知

public boolean contains(Object o) {
    if (o == null) return false;
    for (Node<E> p = first(); p != null; p = succ(p)) {
        E item = p.item;
        if (item != null && o.equals(item))
            return true;
    }
    return false;
}

二、LinkedBlockingQueue原理探究

独占锁实现的有界阻塞队列

类图结构

在这里插入图片描述

takeLock、putLock：分别控制元素出队、入队的原子性；
notEmpty、notFull：条件变量，内部都有一个条件队列用于存放进队和出队时被阻塞的线程（生产——消费者模式）

private final ReentrantLock takeLock = new ReentrantLock();
private final Condition notEmpty = takeLock.newCondition();

private final ReentrantLock putLock = new ReentrantLock();
private final Condition notFull = putLock.newCondition();
// 当前队列元素个数
private final AtomicInteger count = new AtomicInteger();

public LinkedBlockingQueue() {
    this(Integer.MAX_VALUE);
}

public LinkedBlockingQueue(int capacity) {
    if (capacity <= 0) throw new IllegalArgumentException();
    this.capacity = capacity;
    last = head = new Node<E>(null);
}

原理介绍

offer()

public boolean offer(E e) {
    if (e == null) throw new NullPointerException();
    final AtomicInteger count = this.count;
    // 队列满，则丢弃要放入的元素，返回false
    if (count.get() == capacity)
        return false;
    int c = -1;
    Node<E> node = new Node<E>(e);
    final ReentrantLock putLock = this.putLock;
    putLock.lock();
    try {
        if (count.get() < capacity) {
            enqueue(node);
            c = count.getAndIncrement();
            if (c + 1 < capacity)
                notFull.signal();
        }
    } finally {
        putLock.unlock();
    }
    if (c == 0)
    	// 激活notEmpty的条件队列中因调用notEmpty的await方法而被阻塞的线程
        signalNotEmpty();
    return c >= 0;
}

private void enqueue(Node<E> node) {
    last = last.next = node;
}

private void signalNotEmpty() {
    final ReentrantLock takeLock = this.takeLock;
    takeLock.lock();
    try {
        notEmpty.signal();
    } finally {
        takeLock.unlock();
    }
}

put()：队列空闲则直接插入；队列满则阻塞线程，直到有空闲再插入；会对线程中断做出响应

public void put(E e) throws InterruptedException {
    if (e == null) throw new NullPointerException();
    int c = -1;
    Node<E> node = new Node<E>(e);
    final ReentrantLock putLock = this.putLock;
    final AtomicInteger count = this.count;
    putLock.lockInterruptibly();
    try {
        while (count.get() == capacity) {
            notFull.await();
        }
        enqueue(node);
        c = count.getAndIncrement();
        if (c + 1 < capacity)
            notFull.signal();
    } finally {
        putLock.unlock();
    }
    if (c == 0)
        signalNotEmpty();
}

poll()：获取并移除队列头，不阻塞

public E poll() {
    final AtomicInteger count = this.count;
    if (count.get() == 0)
        return null;
    E x = null;
    int c = -1;
    final ReentrantLock takeLock = this.takeLock;
    takeLock.lock();
    try {
        if (count.get() > 0) {
            x = dequeue();
            c = count.getAndDecrement();
            if (c > 1)
                notEmpty.signal();
        }
    } finally {
        takeLock.unlock();
    }
    if (c == capacity)
        signalNotFull();
    return x;
}

private E dequeue() {
    Node<E> h = head;
    Node<E> first = h.next;
    h.next = h; // help GC
    head = first;
    E x = first.item;
    first.item = null;
    return x;
}

private void signalNotFull() {
    final ReentrantLock putLock = this.putLock;
    putLock.lock();
    try {
        notFull.signal();
    } finally {
        putLock.unlock();
    }
}

peek()

public E peek() {
    if (count.get() == 0)
        return null;
    final ReentrantLock takeLock = this.takeLock;
    takeLock.lock();
    try {
        Node<E> first = head.next;
        if (first == null)
            return null;
        else
            return first.item;
    } finally {
        takeLock.unlock();
    }
}

take()：类似poll。不同之处在于，若队列头为空则阻塞，会对线程中断做出响应

public E take() throws InterruptedException {
    E x;
    int c = -1;
    final AtomicInteger count = this.count;
    final ReentrantLock takeLock = this.takeLock;
    takeLock.lockInterruptibly();
    try {
        while (count.get() == 0) {
            notEmpty.await();
        }
        x = dequeue();
        c = count.getAndDecrement();
        if (c > 1)
            notEmpty.signal();
    } finally {
        takeLock.unlock();
    }
    if (c == capacity)
        signalNotFull();
    return x;
}

remove()：删除指定元素

public boolean remove(Object o) {
    if (o == null) return false;
    // 双重加锁
    fullyLock();
    try {
        for (Node<E> trail = head, p = trail.next;
             p != null;
             trail = p, p = p.next) {
            if (o.equals(p.item)) {
                unlink(p, trail);
                return true;
            }
        }
        return false;
    } finally {
        fullyUnlock();
    }
}

void fullyLock() {
    putLock.lock();
    takeLock.lock();
}

void unlink(Node<E> p, Node<E> trail) {
    p.item = null;
    trail.next = p.next;
    if (last == p)
        last = trail;
    // 若队列满，则删除后也不忘记唤醒等待的线程
    if (count.getAndDecrement() == capacity)
        notFull.signal();
}

size()

public int size() {
    return count.get();
}

三、ArrayBlockingQueue原理探究

有界数组方式实现的阻塞队列

类图结构

在这里插入图片描述

public ArrayBlockingQueue(int capacity) {
    this(capacity, false);
}

public ArrayBlockingQueue(int capacity, boolean fair) {
    if (capacity <= 0)
        throw new IllegalArgumentException();
    this.items = new Object[capacity];
    lock = new ReentrantLock(fair);
    notEmpty = lock.newCondition();
    notFull =  lock.newCondition();
}

默认情况下使用ReentrantLock提供的非公平独占锁进行出入队操作

ArrayBlockingQueue原理介绍

offer()：队列尾插入，满则丢弃，不阻塞

public boolean offer(E e) {
    checkNotNull(e);
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        if (count == items.length)
            return false;
        else {
            enqueue(e);
            return true;
        }
    } finally {
        lock.unlock();
    }
}

private void enqueue(E x) {
    final Object[] items = this.items;
    items[putIndex] = x;
    if (++putIndex == items.length)
    	// 因为是单向队列，所以不怕下次offer覆盖已存在的元素，因为只有头移除了count才会减下来，才能进行offer操作
        putIndex = 0;
    count++;
    notEmpty.signal();
}

书上说这里的count不会有内存不可见问题，因为加过锁后的共享变量都是从主存中获取的；释放锁后会把修改的共享变量刷新回主存。

put()：队列尾插入，满则阻塞，会对中断做出响应

public void put(E e) throws InterruptedException {
    checkNotNull(e);
    final ReentrantLock lock = this.lock;
    lock.lockInterruptibly();
    try {
        while (count == items.length)
            notFull.await();
        enqueue(e);
    } finally {
        lock.unlock();
    }
}

poll()：移除头，不阻塞

public E poll() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        return (count == 0) ? null : dequeue();
    } finally {
        lock.unlock();
    }
}

private E dequeue() {
    final Object[] items = this.items;
    E x = (E) items[takeIndex];
    items[takeIndex] = null;
    if (++takeIndex == items.length)
        takeIndex = 0;
    count--;
    if (itrs != null)
        itrs.elementDequeued();
    notFull.signal();
    return x;
}

take()：移除头，队列空则阻塞至队列有元素，对中断做出响应

public E take() throws InterruptedException {
    final ReentrantLock lock = this.lock;
    lock.lockInterruptibly();
    try {
        while (count == 0)
            notEmpty.await();
        return dequeue();
    } finally {
        lock.unlock();
    }
}

peek()：读头元素

public E peek() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        return itemAt(takeIndex); // null when queue is empty
    } finally {
        lock.unlock();
    }
}

final E itemAt(int i) {
    return (E) items[i];
}

size()：加锁是为了从内存读count

public int size() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        return count;
    } finally {
        lock.unlock();
    }
}

ArrayBlockingQueue通过全局独占锁实现了同时只有一个线程进行入队或者出队操作。这个锁的粒度比较大，有点类似于在方法上添加synchronized的意思。相比LinkedBlockingQueue，ArrayBlockingQueue的size操作结果是精确的。

四、PriorityBlockingQueue原理探究

带优先级的无界阻塞队列，每次出队都是返回优先级最高或者最低的元素。其内部是使用平衡二叉树堆实现的，所以直接遍历队列元素不保证有序。默认使用对象的compareTo方法提供比较规则

类图结构

在这里插入图片描述

queue：数组，存放队列元素
size：队列元素个数
allocationSpinLock：自旋锁，用CAS操作保证同时只有一个线程可以扩容队列。状态为0或1；0表示没有进行扩容，1表示正在扩容
comparator：比较器
notEmpty：同来实现take方法阻塞模式。没有notFull是因为这是一个无界队列

private static final int DEFAULT_INITIAL_CAPACITY = 11;

public PriorityBlockingQueue() {
    this(DEFAULT_INITIAL_CAPACITY, null);
}

public PriorityBlockingQueue(int initialCapacity) {
    this(initialCapacity, null);
}

public PriorityBlockingQueue(int initialCapacity,
                             Comparator<? super E> comparator) {
    if (initialCapacity < 1)
        throw new IllegalArgumentException();
    this.lock = new ReentrantLock();
    this.notEmpty = lock.newCondition();
    this.comparator = comparator;
    this.queue = new Object[initialCapacity];
}

默认容量11
默认比较器为null，也就是使用元素的compareTo方法，意味着元素需要实现Comparable接口

原理介绍

offer()

public boolean offer(E e) {
    if (e == null)
        throw new NullPointerException();
    final ReentrantLock lock = this.lock;
    lock.lock();
    int n, cap;
    Object[] array;
    // （1）如果当前元素个数>=队列容量，则扩容
    while ((n = size) >= (cap = (array = queue).length))
        tryGrow(array, cap);
    try {
        Comparator<? super E> cmp = comparator;
        if (cmp == null)
            siftUpComparable(n, e, array);
        else
        	// 自定义比较器
            siftUpUsingComparator(n, e, array, cmp);
		// 将队列元素增加1，并且激活notEmpty的条件队列里面的一个阻塞线程
        size = n + 1;
        notEmpty.signal(); // 激活因调用take()阻塞的线程
    } finally {
        lock.unlock();
    }
    return true;
}

// 扩容和内部建堆
private void tryGrow(Object[] array, int oldCap) {
    lock.unlock(); // 释放锁
    Object[] newArray = null;
    // CAS成功则扩容
    if (allocationSpinLock == 0 &&
        UNSAFE.compareAndSwapInt(this, allocationSpinLockOffset,
                                 0, 1)) {
        try {
        	// oldCap<64则扩容执行oldCap+2，否则扩容50%，最大为MAX_ARRAY_SIZE
            int newCap = oldCap + ((oldCap < 64) ?
                                   (oldCap + 2) : // grow faster if small
                                   (oldCap >> 1));
            if (newCap - MAX_ARRAY_SIZE > 0) {    // possible overflow
                int minCap = oldCap + 1;
                if (minCap < 0 || minCap > MAX_ARRAY_SIZE)
                    throw new OutOfMemoryError();
                newCap = MAX_ARRAY_SIZE;
            }
            if (newCap > oldCap && queue == array)
                newArray = new Object[newCap];
        } finally {
            allocationSpinLock = 0;
        }
    }
    // 第一个线程CAS成功后，第二个线程会进入这段代码，然后第二个线程让出CPU，尽量让第一个线程获取锁，但是这得不到保证
    if (newArray == null) // back off if another thread is allocating
        Thread.yield();
    lock.lock();
    if (newArray != null && queue == array) {
        queue = newArray;
        System.arraycopy(array, 0, newArray, 0, oldCap);
    }
}

为啥扩容前要先释放锁？：扩容需要花时间，若一直持有，其他线程无法进行出、入队操作，大大降低了并发性。因此使用CAS控制只有一个线程可以扩容，并再扩容前释放锁让其他线程可以进行出、入队操作。
spinlock锁使用CAS控制只有一个线程可以进行扩容，CAS失败的线程会调用Thread.yield()让出CUP，目的是让扩容线程扩容后优先调用lock.lock()重新获取锁，但是这得不到保证。有可能yield的线程在扩容完成前已经退出，并且获取锁，但是扩容还在继续，所以会再次执行代码（1），重新释放锁给扩容现成获取（自旋）。
扩容线程扩容完毕后会重置自旋变量allocationSpinLock为0，没有使用Unsafe是因为前面保证了只有一个线程可以扩容。之后获取锁，并复制当前queue里面的元素到新数组

建堆算法

private static <T> void siftUpComparable(int k, T x, Object[] array) {
    Comparable<? super T> key = (Comparable<? super T>) x;
    while (k > 0) {
        int parent = (k - 1) >>> 1;
        Object e = array[parent];
        if (key.compareTo((T) e) >= 0)
            break;
        array[k] = e;
        k = parent;
    }
    array[k] = key;
}

poll()获取队列内部堆树的根节点元素

public E poll() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        return dequeue();
    } finally {
        lock.unlock();
    }
}

private E dequeue() {
    int n = size - 1;
    if (n < 0)
        return null;
    else {
	    // 获取队头元素
        Object[] array = queue;
        E result = (E) array[0];
		// 获取队尾元素，并赋值为null
        E x = (E) array[n];
        array[n] = null;
        Comparator<? super E> cmp = comparator;
        if (cmp == null)
            siftDownComparable(0, x, array, n);
        else
            siftDownUsingComparator(0, x, array, n, cmp);
        size = n;
        return result;
    }
}

private static <T> void siftDownComparable(int k, T x, Object[] array,
                                           int n) {
    if (n > 0) {
        Comparable<? super T> key = (Comparable<? super T>)x;
        int half = n >>> 1;           // loop while a non-leaf
        while (k < half) {
            int child = (k << 1) + 1; // assume left child is least
            Object c = array[child];
            int right = child + 1;
            if (right < n &&
                ((Comparable<? super T>) c).compareTo((T) array[right]) > 0)
                c = array[child = right];
            if (key.compareTo((T) c) <= 0)
                break;
            array[k] = c;
            k = child;
        }
        array[k] = key;
    }
}

put()：由于是无界队列，故不阻塞

public void put(E e) {
    offer(e); // never need to block
}

take()：获取队列内部堆树的根节点元素，若队列空则阻塞

public E take() throws InterruptedException {
    final ReentrantLock lock = this.lock;
    lock.lockInterruptibly();
    E result;
    try {
        while ( (result = dequeue()) == null)
            notEmpty.await();
    } finally {
        lock.unlock();
    }
    return result;
}

size()：

public int size() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        return size;
    } finally {
        lock.unlock();
    }
}

四、DelayQueue原理探究

无界阻塞延迟队列，队列中每个元素都有个过期时间，当从队列获取元素时，只有过期元素才会出队列。队列头元素是最快要过期的元素

DelayQueue类图结构

在这里插入图片描述

内部使用PriorityQueue存放数据
使用ReentrantLock实现线程同步。
队列里的元素要实现Delayed接口（继承Comparable接口）
available与lock锁时对应的，其目的是为了实现线程间同步
leader变量的使用基于Leader-Follower模式的变体，用于尽量减少不必要的线程等待。当一个线程调用队列的take方法变为leader线程后，它会调用条件变量available.awaitNanos(delay)等待，但是其他线程（follwer线程）则会调用available.await()进行无线等待。leader线程延迟时间过期后，会推出take方法，并通过调用available.signal()方法唤醒一个follwer线程，被唤醒的follwer线程被选举为新的leader线程

主要方法原理讲解

offer()

public boolean offer(E e) {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        q.offer(e);
        if (q.peek() == e) {
            leader = null;
            available.signal();
        }
        return true;
    } finally {
        lock.unlock();
    }
}

take()：获取并移除队列里面延迟时间过期的元素，若队列没有过期元素则等待

public E take() throws InterruptedException {
    final ReentrantLock lock = this.lock;
    lock.lockInterruptibly();
    try {
        for (;;) {
        	// 获取但步移除队首
            E first = q.peek();
            if (first == null)
                available.await();
            else {
                long delay = first.getDelay(NANOSECONDS);
                if (delay <= 0)
                    return q.poll();
                first = null; // don't retain ref while waiting
                if (leader != null)
                    available.await();
                else {
                    Thread thisThread = Thread.currentThread();
                    leader = thisThread;
                    try {
                        available.awaitNanos(delay);
                    } finally {
                        if (leader == thisThread)
                            leader = null;
                    }
                }
            }
        }
    } finally {
        if (leader == null && q.peek() != null)
            available.signal();
        lock.unlock();
    }
}

poll()：获取并移除队头过期元素，若没有过期元素则返回null

public E poll() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        E first = q.peek();
        if (first == null || first.getDelay(NANOSECONDS) > 0)
            return null;
        else
            return q.poll();
    } finally {
        lock.unlock();
    }
}

size()：计算队列元素，包括不过期和过期的

public int size() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        return q.size();
    } finally {
        lock.unlock();
    }
}

_大聪明

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
并发编程笔记——第七章 Java并发包中并发队列原理剖析

JDK中提供了一系列场景的并发安全队列。总体来说，按照实现方式的不同可分为阻塞队列和非阻塞队列，前者使用锁实现，后者则使用CAS非阻塞算法实现一、ConcurrentLinkedQueue原理探究线程安全的无界非阻塞队列，底层数据结构使用单向链表实现，入队出队操作使用CAS来实现线程安全类图结构原理介绍offer操作：在队列末尾添加元素public boolean offer(E e) { checkNotNull(e); // 构造Node节点，构造函数内部调用uns.
复制链接

扫一扫