关于LinkedBlockingQueue高并发瓶颈优化

在线程数量比较多的时候,LinkedBlockingQueue.take() 并发性能并不是特别优秀。
在实际项目中,有个场景一个LinkedBlockingQueue后面挂了60个线程,数据量大,线程量也大,但通过堆栈信息查看,大约有五十多个线程处于WAITING状态, 都卡在LinkedBlockingQueue.take() 处。
为提高性能,细化锁粒度。写了下面一个类,时间紧急只实现了 add, offer, take接口。


import java.util.Collection;
import java.util.Iterator;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * 本质上就是将一个大队列,分隔成多个子队列,这样降低锁粒度
 * 但是外在的消费端线程数量有一定要求,就是必须大于等于子队列,否则会造成数据无法被消费的尴尬情况
 * @param <T>
 */
public class FastLinkedBlockingQueue<T> implements BlockingQueue<T> {

    /**
     * 计算长度
     */
    private AtomicInteger size = new AtomicInteger();

    /**
     * 子队列数量
     */
    private int factor;

    /**
     * 子队列
     */
    private LinkedBlockingQueue<T>[] children;

    /**
     * 消费该队列的线程数量必须大于等于factor,否则会造成数据无法被消费
     * @param factor
     */
    public FastLinkedBlockingQueue(int factor){
        this.factor = factor;
        this.children = new LinkedBlockingQueue[factor];
        for (int i = 0; i < factor; i++) {
            children[i] = new LinkedBlockingQueue<>();
        }
    }

    @Override
    public boolean add(T t) {
        int i = size.incrementAndGet();
        return this.children[i%factor].add(t);
    }

    @Override
    public boolean offer(T t) {
        int i = size.incrementAndGet();
        return this.children[i%factor].offer(t);
    }

    /**
     * 保存每个线程消费子队列的下标,如为空,表示当前线程是第一次消费此队列,
     * 那么就需要如下算法确定此线程到底消费哪个子队列
     * 先由consumerCount按序增加,然后对子序列的个数进行取模
     */
    private ThreadLocal<Integer> threadIndex = new ThreadLocal<>();
    private AtomicInteger consumerCount = new AtomicInteger(-1);

    @Override
    public T take() throws InterruptedException {
        Integer index = threadIndex.get();
        //为空表示第一次取数据
        if (index == null) {
            //消费者+1
            index = consumerCount.incrementAndGet();
            // 对子序列长度取模
            index = index % factor;
            // 设置当前线程消费的子队列下标,下次消费直接根据下标找到子队列
            threadIndex.set(index);
        }
        T take = children[index].take();
        size.decrementAndGet();
        return take;
    }

    @Override
    public int size() {
        return size.get();
    }

    @Override
    public T remove() {return null;}
    @Override
    public T poll() {return null;}
    @Override
    public T element() {return null;}
    @Override
    public T peek() {return null;}
    @Override
    public void put(T t) throws InterruptedException {}
    @Override
    public boolean offer(T t, long timeout, TimeUnit unit) throws InterruptedException {return false;}
    @Override
    public T poll(long timeout, TimeUnit unit) throws InterruptedException {return null;}
    @Override
    public int remainingCapacity() {return 0;}
    @Override
    public boolean remove(Object o) {return false;}
    @Override
    public boolean containsAll(Collection<?> c) {return false;}
    @Override
    public boolean addAll(Collection<? extends T> c) {return false;}
    @Override
    public boolean removeAll(Collection<?> c) {return false;}
    @Override
    public boolean retainAll(Collection<?> c) {return false;}
    @Override
    public void clear() {}
    @Override
    public boolean isEmpty() {return false;}
    @Override
    public boolean contains(Object o) {return false;}
    @Override
    public Iterator<T> iterator() {return null;}
    @Override
    public Object[] toArray() {return new Object[0];}
    @Override
    public <T1> T1[] toArray(T1[] a) {return null;}
    @Override
    public int drainTo(Collection<? super T> c) {return 0;}
    @Override
    public int drainTo(Collection<? super T> c, int maxElements) {return 0;}
}

性能测试


import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.atomic.AtomicInteger;

public class Test {
    static BlockingQueue<Integer> queue = new FastLinkedBlockingQueue<>(15);
//    static BlockingQueue<Integer> queue = new LinkedBlockingQueue<>();
    static CountDownLatch latch = new CountDownLatch(1);
    static int total = 10000000;
    public static void main(String[] args) throws InterruptedException {
        for (int i = 0; i < total; i++) {
            queue.offer(i);
        }
        System.out.println("放置完成");
        long start = System.currentTimeMillis();
        Comsumer comsumer = new Comsumer();
        for (int i = 0; i < 30; i++) {
            new Thread(comsumer).start();
        }
        latch.await();
        System.out.println("耗时:" + (System.currentTimeMillis() - start));
    }
}

class Comsumer implements Runnable{

    AtomicInteger count = new AtomicInteger();
    @Override
    public void run() {
        while (true) {
            try {
                Test.queue.take();
                int num = count.incrementAndGet();
                if (num == Test.total) {
                    Test.latch.countDown();
                }
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}


在线程数量较小的情况下,性能差别不大,但当线程数量较大时,性能差距就不一样了

  • 2
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值