REDIS (13) 并发使用Jedis原理分析

1. jedis的部分源码 了解connectTimeout和socketTimeOut参数

我们先从Jedis的源代码入手了解Jedis

JedisPool pool = new JedisPool(poolConfig, host, port, timeout, psw, database);

找到JedisFactory.makeObject()。上面的timeout参数传给了下面connectionTimeOut和soTimeOut，最终这两个参数传给了JSE的Socket类的实例。

@Override
  public PooledObject<Jedis> makeObject() throws Exception {
    final HostAndPort hostAndPort = this.hostAndPort.get();
    final Jedis jedis = new Jedis(hostAndPort.getHost(), hostAndPort.getPort(), connectionTimeout,
        soTimeout);

    jedis.connect();
    if (null != this.password) {
      jedis.auth(this.password);
    }
    if (database != 0) {
      jedis.select(database);
    }
    if (clientName != null) {
      jedis.clientSetname(clientName);
    }

    return new DefaultPooledObject<Jedis>(jedis);
  }

最后我们找到

public void connect() {
    if (!isConnected()) {
      try {
        socket = new Socket();
        // ->@wjw_add
        socket.setReuseAddress(true);
        socket.setKeepAlive(true); // Will monitor the TCP connection is
        // valid
        socket.setTcpNoDelay(true); // Socket buffer Whetherclosed, to
        // ensure timely delivery of data
        socket.setSoLinger(true, 0); // Control calls close () method,
        // the underlying socket is closed
        // immediately
        // <-@wjw_add

        socket.connect(new InetSocketAddress(host, port), connectionTimeout);
        socket.setSoTimeout(soTimeout);
        outputStream = new RedisOutputStream(socket.getOutputStream());
        inputStream = new RedisInputStream(socket.getInputStream());
      } catch (IOException ex) {
        broken = true;
        throw new JedisConnectionException(ex);
      }
    }
  }

2.并发测试: Jedis线程不安全

JedisPool pool = new JedisPool(poolConfig, "123.56.XX.XX", 6379, 0, "foobared", 13);
        AtomicInteger n =new AtomicInteger(1000);
        String timeStr = TimeTool.formatTime(System.currentTimeMillis(), "yyyy_MM_dd_HH_mm_ss_SSS");
        ThreadTool.sleep(100);
        try (Jedis jedis = pool.getResource()) {
            while(n.decrementAndGet()>0){
                new Thread(()->{
                    System.out.print(".");
                    Long rst = jedis.incr(timeStr);
                    LogCore.BASE.info("test={}", rst);
                }).start();
            }

        } catch (Exception e) {
            LogCore.BASE.info("manager fatal err,", e);
        }

报的异常:
Exception in thread "Thread-925" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Broken pipe (Write failed)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:98)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:78)
    at redis.clients.jedis.Connection.sendCommand(Connection.java:101)
    at redis.clients.jedis.BinaryClient.incr(BinaryClient.java:217)
    at redis.clients.jedis.Client.incr(Client.java:135)
    at redis.clients.jedis.Jedis.incr(Jedis.java:538)
    at com.sincetimes.website.redis.jedis.excample.ConcurrentTest.lambda$0(ConcurrentTest.java:34)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Broken pipe (Write failed)
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
    at redis.clients.util.RedisOutputStream.flushBuffer(RedisOutputStream.java:31)
    at redis.clients.util.RedisOutputStream.write(RedisOutputStream.java:38)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:84)
    ... 7 more
Exception in thread "Thread-924" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Broken pipe (Write failed)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:98)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:78)
    at redis.clients.jedis.Connection.sendCommand(Connection.java:101)
    at redis.clients.jedis.BinaryClient.incr(BinaryClient.java:217)
    at redis.clients.jedis.Client.incr(Client.java:135)
    at redis.clients.jedis.Jedis.incr(Jedis.java:538)
    at com.sincetimes.website.redis.jedis.excample.ConcurrentTest.lambda$0(ConcurrentTest.java:34)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Broken pipe (Write failed)
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
    at redis.clients.util.RedisOutputStream.flushBuffer(RedisOutputStream.java:31)
    at redis.clients.util.RedisOutputStream.write(RedisOutputStream.java:38)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:84)
    ... 7 more

3.并发测试 JedisPool是线程安全的 了解maxTotal和maxWaitMillis参数

JedisPool pool = new JedisPool(poolConfig, "123.56.XX.XX", 6379, 0, "foobared", 13);
AtomicInteger n =new AtomicInteger(1000);
        ThreadTool.sleep(100);
        while(n.decrementAndGet()>0){
            new Thread(()->{
                try (Jedis jedis = pool.getResource()) {
                    LogCore.BASE.info("test={}", jedis.set("test", "n"+n.get()));

                } catch (Exception e) {
                    LogCore.BASE.info("manager fatal err,", e);
                }
            }).start();
        }

        pool.close();

4.JedisPool的实现

4.1 从连接池获取Jedis

T org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(long borrowMaxWaitMillis)

 public T borrowObject(long borrowMaxWaitMillis) throws Exception {
        assertOpen();

        AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getRemoveAbandonedOnBorrow() &&
                (getNumIdle() < 2) &&
                (getNumActive() > getMaxTotal() - 3) ) {
            removeAbandoned(ac);
        }

        PooledObject<T> p = null;

        // Get local copy of current config so it is consistent for entire
        // method execution
        boolean blockWhenExhausted = getBlockWhenExhausted();

        boolean create;
        long waitTime = System.currentTimeMillis();

        while (p == null) {
            create = false;
            if (blockWhenExhausted) {
                p = idleObjects.pollFirst();
                if (p == null) {
                    p = create();
                    if (p != null) {
                        create = true;
                    }
                }
                if (p == null) {
                    if (borrowMaxWaitMillis < 0) {
                        p = idleObjects.takeFirst();
                    } else {
                        p = idleObjects.pollFirst(borrowMaxWaitMillis,
                                TimeUnit.MILLISECONDS);
                    }
                }
                if (p == null) {
                    throw new NoSuchElementException(
                            "Timeout waiting for idle object");
                }
                if (!p.allocate()) {
                    p = null;
                }
            } else {
                p = idleObjects.pollFirst();
                if (p == null) {
                    p = create();
                    if (p != null) {
                        create = true;
                    }
                }
                if (p == null) {
                    throw new NoSuchElementException("Pool exhausted");
                }
                if (!p.allocate()) {
                    p = null;
                }
            }

            if (p != null) {
                try {
                    factory.activateObject(p);
                } catch (Exception e) {
                    try {
                        destroy(p);
                    } catch (Exception e1) {
                        // Ignore - activation failure is more important
                    }
                    p = null;
                    if (create) {
                        NoSuchElementException nsee = new NoSuchElementException(
                                "Unable to activate object");
                        nsee.initCause(e);
                        throw nsee;
                    }
                }
                if (p != null && (getTestOnBorrow() || create && getTestOnCreate())) {
                    boolean validate = false;
                    Throwable validationThrowable = null;
                    try {
                        validate = factory.validateObject(p);
                    } catch (Throwable t) {
                        PoolUtils.checkRethrow(t);
                        validationThrowable = t;
                    }
                    if (!validate) {
                        try {
                            destroy(p);
                            destroyedByBorrowValidationCount.incrementAndGet();
                        } catch (Exception e) {
                            // Ignore - validation failure is more important
                        }
                        p = null;
                        if (create) {
                            NoSuchElementException nsee = new NoSuchElementException(
                                    "Unable to validate object");
                            nsee.initCause(validationThrowable);
                            throw nsee;
                        }
                    }
                }
            }
        }

        updateStatsBorrow(p, System.currentTimeMillis() - waitTime);

        return p.getObject();
    }

其中borrowMaxWaitMillis的就是maxWaitMillis默认值为-1
也就是上面的一大堆我们只需要关心如下的代码：

if (borrowMaxWaitMillis < 0) {
    p = idleObjects.takeFirst();
} else {
    p = idleObjects.pollFirst(borrowMaxWaitMillis,
TimeUnit.MILLISECONDS);
}
if (p == null) {
    throw new NoSuchElementException("Timeout waiting for idle object");
}

idleObjects是个阻塞队列,而且是双向的也就是Deque而不是Queue
(org.apache.commons.pool2.impl.LinkedBlockingDeque
这个类是apache的与Java的并发包下的LinkedBlockingDeque类似
至于为什么使用deque因为Apache的这套连接池是可以控制队列是先进先出FIFO还是先进后出LIFO的,JedisPoolConfig可以用.setLifo(lifo);来控制。

apache.commons.pool2.impl.LinkedBlockingDeque	java.util.concurrent.LinkedBlockingDeque
E takeFirst() throws InterruptedException	E takeFirst() throws InterruptedException
E pollFirst(long timeout, TimeUnit unit)	E pollFirst(long timeout, TimeUnit unit)

接口一样我们再比较下源码
jse:

public E takeFirst() throws InterruptedException {
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
            E x;
            while ( (x = unlinkFirst()) == null)
                notEmpty.await();
            return x;
        } finally {
            lock.unlock();
        }
    }
public E pollFirst(long timeout, TimeUnit unit)
        throws InterruptedException {
        long nanos = unit.toNanos(timeout);
        final ReentrantLock lock = this.lock;
        lock.lockInterruptibly();
        try {
            E x;
            while ( (x = unlinkFirst()) == null) {
                if (nanos <= 0)
                    return null;
                nanos = notEmpty.awaitNanos(nanos);
            }
            return x;
        } finally {
            lock.unlock();
        }
    }

apache

 public E takeFirst() throws InterruptedException {
        lock.lock();
        try {
            E x;
            while ( (x = unlinkFirst()) == null) {
                notEmpty.await();
            }
            return x;
        } finally {
            lock.unlock();
        }
    }
 public E pollFirst(long timeout, TimeUnit unit)
        throws InterruptedException {
        long nanos = unit.toNanos(timeout);
        lock.lockInterruptibly();
        try {
            E x;
            while ( (x = unlinkFirst()) == null) {
                if (nanos <= 0) {
                    return null;
                }
                nanos = notEmpty.awaitNanos(nanos);
            }
            return x;
        } finally {
            lock.unlock();
        }
    }

几乎一样
也就是当jedis从队列获取不到的时候，会阻塞直到队列非空。如果我们设置了maxWaitMillis参数，而jedispool已满(达到maxTotal)pool.getResouce在这个时间迟迟等不到其他的Jedis返回给连接池时就会报如下异常：

Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
    at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:449)
    at redis.clients.util.Pool.getResource(Pool.java:48)
    ... 3 common frames omitted

4.2 创建Jedis

可见localMaxTotal参数控制了最多生成的连接数

private PooledObject<T> create() throws Exception {
        int localMaxTotal = getMaxTotal();
        long newCreateCount = createCount.incrementAndGet();
        if (localMaxTotal > -1 && newCreateCount > localMaxTotal ||
                newCreateCount > Integer.MAX_VALUE) {
            createCount.decrementAndGet();
            return null;
        }

        final PooledObject<T> p;
        try {
            p = factory.makeObject();
        } catch (Exception e) {
            createCount.decrementAndGet();
            throw e;
        }

        AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getLogAbandoned()) {
            p.setLogAbandoned(true);
        }

        createdCount.incrementAndGet();
        allObjects.put(new IdentityWrapper<T>(p.getObject()), p);
        return p;
    }

总之，Jedis线程不安全，JedisPool线程安全。其最大的连接数和获取等待时间是可以设置上限的，用完后连接变为空闲，活跃的连接(active)和空闲的(idel)不会超过最大连接数(total)