多线程+阻塞队列实现生产者-消费者模型获取队列数据问题

最近在研究服务端推送技术，使用后台轮询的方式hold住连接，当有符合条件的数据产生时推送数据到前台。业务定期产生数据放到队列中，连接循环从队列中取数据，由于存在多个连接从同一队列中取数据的情况，考虑到是否会有数据不能被及时消费，或者队首数据一直不能被消费造成死循环的情况。

在网上查了一下多线程时间片段分配，如果没有设定优先级，各线程应该是有均等的机会被执行的。本着严谨的态度，通过程序做了如下测试。

生产线程

public class QueueDataGenerator implements Runnable{

  public static BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
  public static boolean run = false;

  @Override
  public void run() {
    run = true;
    Random r = new Random();
    SimpleDateFormat format = new SimpleDateFormat("mm:ss");
    for (int i = 0; i < 100; i++) {
      String v = String.valueOf(r.nextInt(10));
      try {
        String ms = format.format(new Date());
        System.out.println("put data '"+ v +"' to queue. CurrentTime:"+ ms);
        queue.put(v);
        Thread.sleep(2000);
      } catch (InterruptedException e) {
        e.printStackTrace();
      }
    }
    run = false;
  }
}

循环向队列中放入数据，每次放入一个0-9的随机整数

消费线程

public class Polling implements Runnable{

  private String id;
  /**
   * .
   * @param id
   */
  public Polling(String id) {
    this.id = id;
  }

  @Override
  public void run() {
    SimpleDateFormat format = new SimpleDateFormat("mm:ss");
    while(QueueDataGenerator.run){
      String v = QueueDataGenerator.queue.peek();
      if (id.equals(v)) {
        String ms = format.format(new Date());
        System.out.println("Polling with id '"+ id +"' match. CurrentTime:"+ ms);
        QueueDataGenerator.queue.remove(v);
      }
      //try {
        //Thread.sleep(1000);
      //} catch (InterruptedException e) {
        //e.printStackTrace();
      //}
    }
  }
}

通过QueueDataGenerator.run判断是否处在队列数据生产周期，若是则循环从队列中取数据，peek()方法不会移除队首数据，每个消费者有自己的标识id，若队首数据与id相等，则移除数据，表明数据被成功消费。

下面是测试类

public class Test {

  public static void main(String[] args) {
    QueueDataGenerator dataGenerator = new QueueDataGenerator();
    Thread t1 = new Thread(dataGenerator);
    t1.start();

    for (int i = 0; i < 10; i++) {
      String id = String.valueOf(i);
      Polling polling = new Polling(id);
      Thread t = new Thread(polling);
      t.start();
    }

  }
}

开启生产线程，然后循环开启10个消费线程。

为方便观察数据变化，打印了时间（分/秒），部分测试结果如下：

put data '5' to queue. CurrentTime:14:33
Polling with id '5' match. CurrentTime:14:33
put data '6' to queue. CurrentTime:14:35
Polling with id '6' match. CurrentTime:14:35
put data '8' to queue. CurrentTime:14:37
Polling with id '8' match. CurrentTime:14:37
put data '1' to queue. CurrentTime:14:39
Polling with id '1' match. CurrentTime:14:39
put data '9' to queue. CurrentTime:14:41
Polling with id '9' match. CurrentTime:14:41
put data '7' to queue. CurrentTime:14:43
Polling with id '7' match. CurrentTime:14:43

因为生产线程是2秒产生一个数据，可以看到每个数据都能被对应的线程消费，不存在线程“争抢”影响其他消费线程执行的情况。

进一步测试消费线程执行时间过长是否会对结果产生影响，打开注释代码，
让消费线程sleep一秒：

put data '9' to queue. CurrentTime:19:27
Polling with id '9' match. CurrentTime:19:28
put data '1' to queue. CurrentTime:19:29
Polling with id '1' match. CurrentTime:19:30
put data '5' to queue. CurrentTime:19:31
Polling with id '5' match. CurrentTime:19:31
put data '4' to queue. CurrentTime:19:33
Polling with id '4' match. CurrentTime:19:33
put data '8' to queue. CurrentTime:19:35
Polling with id '8' match. CurrentTime:19:35

sleep 3秒：

put data '2' to queue. CurrentTime:25:24
put data '9' to queue. CurrentTime:25:26
Polling with id '2' match. CurrentTime:25:27
put data '7' to queue. CurrentTime:25:28
Polling with id '9' match. CurrentTime:25:30
put data '6' to queue. CurrentTime:25:30
put data '3' to queue. CurrentTime:25:32
Polling with id '7' match. CurrentTime:25:33
Polling with id '6' match. CurrentTime:25:33
put data '9' to queue. CurrentTime:25:34
Polling with id '3' match. CurrentTime:25:36
put data '8' to queue. CurrentTime:25:36
put data '9' to queue. CurrentTime:25:38
Polling with id '9' match. CurrentTime:25:39

可以看出，执行时间长也只是被消费的时间延长相应时间（随着执行时间间隔可能越来越长），并不会造成某些数据不能被消费。一般基于实时性考虑，不会使消费线程执行时间过长，可以将复杂业务逻辑放到生产线程中，消费线程直接拿到可以展示的结果数据。