redis集群客户端JedisCluster优化 - 管道(pipeline)模式支持

Redis在3.0版正式引入了集群这个特性,扩展变得非常简单。然而当你开心的升级到3.0后,却发现有些很好用的功能现在工作不了了, 比如我们今天要聊的pipeline功能。

我们知道,普通的情况下,redis client与server之间采用的是请求应答的模式,即:

Client: command1 
Server: response1 
Client: command2 
Server: response2 

在这种情况下,如果要完成10个命令,则需要20次交互才能完成。因此,即使redis处理能力很强,仍然会受到网络传输影响,导致吞吐上不去。而在管道模式下,多个请求变成这样:

Client: command1,command2… 
Server: response1,response2…

在这种情况下,完成命令只需要2次交互。这样网络传输上能够更加高效,加上redis本身强劲的处理能力,是不是有一种飞一样的感觉。听到这里有没有去优化应用的冲动? 然而到了cluster模式下,这样的功能并不支持。 下面我们先来分析下,是什么原因导致redis cluter没办法支持管道模式。首先需要了解集群下的几个特性:

  • 1、集群将空间分拆为16384个槽位(slot),每一个节点负责其中一些槽位。迁移时对整个slot迁移
  • 2、节点添加,或宕机或不可达的情况下可以正常使用
  • 3、不存在中心或者代理节点, 每个节点中都包含集群所有的节点信息
  • 4、集群中的节点不会代理请求:即如果client将命令发送到错误的节点上,操作会失败,但会返回”-MOVED”或”-ASK”,供client进行永久或临时的节点切换

以上信息中第3、4点信息比较重要。

我们先来看第3点,由于每个节点都包含所有的节点信息,因此client连接任一节点都可以获取整个集群的信息,这样我们在配置JedisCluster时只需要配置其中一部分节点的信息就可以(配置多个是为了高可用)。对应的获取集群命令为:cluster nodes

127.0.0.1:9380> cluster nodes 
b6d0cfe64dbae9590e6fc4c5a8e309debcbe0529 127.0.0.1:9380 myself,master - 0 0 2 connected 5461-10922 
b9e5592558aae0f28c79c3750b264d5b2530f6a4 127.0.0.1:9381 master - 0 1466758609932 3 connected 10923-16383 
b40095eb2023653eaea5b7b4e242a77a7817889a 127.0.0.1:9379 master - 0 1466758608932 1 connected 0-5460

每一行代表一个节点的信息,这里共三个节点(测试用,没有建slave节点),依次的信息为:

{id} {ip:port} {flags如master/slave} {master id} {ping-sent} {pong-recv} {config-epoch} {link-state} {slot} {slot} … {slot} 
参考: http://redis.io/commands/cluster-nodes

可以看到每个节点对应的slot信息都在这里,{slot}格式一般是{begin}-{end}(如0-5460),表示从{begin}到{end}的所有slot都在当前节点中。因此我们可以通过slot找到对应机器的ip:port。 注意,新版client中使用cluster slots获取对应数据 , 参考: http://redis.io/commands/cluster-slots 。

再来看第4点,由第3点可以知道client可以通过获取所有节点信息,根据key计算得到对应的slot后可以找到对应的节点。所以说在节点稳定(没有增减)的情况下,客户端可以一直用缓存的集群信息来发起各种命令。然而,如果节点发生变更客户端是否能够立即感知? 目前的client JedisCluster是无法感知的,他是通过执行命令后, 服务端返回的“-MOVED”信息感知节点的变化,并以此来刷新缓存信息。

了解以上信息以后,JedisCluster为什么不支持pipeline就比较清晰了。 因为pipeline模式下命令将被缓存到对应的连接(OutputStream)上,而在真正向服务端发送数据时,节点可能发生了改变,数据就可能发向了错误的节点,这导致批量操作失败,而要处理这种失败是非常复杂的。至少目前JedisCluster并未提供这样的机制。(对于单key来说,在发生这种情况的时候,进行简单的节点数据刷新+重新发送当前命令来重试)。

看到这里,你可能会感到沮丧(我猿类如此不易,且行且珍惜)。这里提供一个简单的思路,你可以根据单key的逻辑,如果某些key遇到”-MOVE”或”-ASK”则重试。 根据这个思路,你需要按顺序记录所有的命令,每次执行完成后找出异常的数据,刷新节点信息后重试,最终将重试(可能有多次)获取到的结果根据顺序信息插入返回列表。对于重试多次依然失败的数据,交由业务处理。思路很简单,然而redis命令太多了,要对PipelineBase的每个方法都这样改造,我不想(因为我懒呀),而且估计坑也很多,所以这个只能靠你自己去搞了。

下面我说下针对我们的业务做的一个JedisCluster pipeline实现。对应的业务有以下特点: 
数据为每隔一段时间全量导入redis集群,数据量约xx万(xx较大) 
导入任务为后台执行,可重试,最终如果有部分失败可接受 
集群相对较稳定,不会频繁的加减机器 
在线业务不使用该api

下面是该类的源码(jdk1.7),如在以上这几个条件下有问题,可以一起交流: 
(github地址: https://github.com/youaremoon/jedis-ext)

<code class="language-java hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">package</span> com.yam.common.redis;

<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.io.Closeable;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.io.IOException;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.lang.reflect.Field;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.ArrayList;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.HashMap;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.HashSet;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.LinkedList;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.List;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.Map;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.Queue;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> java.util.Set;

<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> javax.annotation.concurrent.NotThreadSafe;

<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> org.slf4j.Logger;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> org.slf4j.LoggerFactory;

<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.BinaryJedisCluster;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.Client;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.HostAndPort;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.Jedis;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.JedisCluster;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.JedisClusterConnectionHandler;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.JedisClusterInfoCache;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.JedisPool;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.JedisSlotBasedConnectionHandler;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.PipelineBase;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.exceptions.JedisMovedDataException;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.jedis.exceptions.JedisRedirectionException;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.util.JedisClusterCRC16;
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">import</span> redis.clients.util.SafeEncoder;

<span class="hljs-javadoc" style="color:#8800;box-sizing: border-box;">/**
 * 在集群模式下提供批量操作的功能。 <br/>
 * 由于集群模式存在节点的动态添加删除,且client不能实时感知(只有在执行命令时才可能知道集群发生变更),
 * 因此,该实现不保证一定成功,建议在批量操作之前调用 refreshCluster() 方法重新获取集群信息。<br />
 * 应用需要保证不论成功还是失败都会调用close() 方法,否则可能会造成泄露。<br/>
 * 如果失败需要应用自己去重试,因此每个批次执行的命令数量需要控制。防止失败后重试的数量过多。<br />
 * 基于以上说明,建议在集群环境较稳定(增减节点不会过于频繁)的情况下使用,且允许失败或有对应的重试策略。<br />
 * 
 * 
 *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @author</span> youaremoon
 *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @version</span>
 *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @since</span> Ver 1.1
 */</span>
<span class="hljs-annotation" style="color:#9b859d;box-sizing: border-box;">@NotThreadSafe</span>
<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">class</span> <span class="hljs-title" style="color:#66066;box-sizing: border-box;">JedisClusterPipeline</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">extends</span> <span class="hljs-title" style="color:#66066;box-sizing: border-box;">PipelineBase</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">implements</span> <span class="hljs-title" style="color:#66066;box-sizing: border-box;">Closeable</span> {</span>

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">final</span> Logger LOGGER = LoggerFactory.getLogger(JedisClusterPipeline.class);

    <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 部分字段没有对应的获取方法,只能采用反射来做</span>
    <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 你也可以去继承JedisCluster和JedisSlotBasedConnectionHandler来提供访问接口</span>
    <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 我没这样做是因为懒</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">final</span> Field FIELD_CONNECTION_HANDLER;
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">final</span> Field FIELD_CACHE; 
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> {
        FIELD_CONNECTION_HANDLER = getField(BinaryJedisCluster.class, <span class="hljs-string" style="color:#0880;box-sizing: border-box;">"connectionHandler"</span>);
        FIELD_CACHE = getField(JedisClusterConnectionHandler.class, <span class="hljs-string" style="color:#0880;box-sizing: border-box;">"cache"</span>);
    }

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> JedisSlotBasedConnectionHandler connectionHandler;
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> Queue<Client> clients = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> LinkedList<Client>();   <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 根据顺序存储每个命令对应的Client</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> Map<JedisPool, Jedis> jedisMap = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> HashMap<>();   <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 用于缓存连接</span>

    <span class="hljs-javadoc" style="color:#8800;box-sizing: border-box;">/**
     * 根据jedisCluster实例生成对应的JedisClusterPipeline
     *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @param</span> 
     *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @return</span>
     */</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> JedisClusterPipeline <span class="hljs-title" style="box-sizing: border-box;">pipelined</span>(JedisCluster jedisCluster) {
        JedisClusterPipeline pipeline = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> JedisClusterPipeline();
        pipeline.setJedisCluster(jedisCluster);
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> pipeline;
    }

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">setJedisCluster</span>(JedisCluster jedis) {
        JedisSlotBasedConnectionHandler ch = getValue(jedis, FIELD_CONNECTION_HANDLER);
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">null</span> == ch) {
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">throw</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> RuntimeException(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"cannot get JedisSlotBasedConnectionHandler from JedisCluster"</span>);
        }

        connectionHandler = ch;
    }

    <span class="hljs-javadoc" style="color:#8800;box-sizing: border-box;">/**
     * 刷新集群信息,当集群信息发生变更时调用
     *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @param</span> 
     *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @return</span>
     */</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">refreshCluster</span>() {
        connectionHandler.renewSlotCache();
    }

    <span class="hljs-javadoc" style="color:#8800;box-sizing: border-box;">/**
     * 同步读取所有数据. 与syncAndReturnAll()相比,sync()只是没有对数据做反序列化
     */</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">sync</span>() {
        innerSync(<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">null</span>);
    }

    <span class="hljs-javadoc" style="color:#8800;box-sizing: border-box;">/**
     * 同步读取所有数据 并按命令顺序返回一个列表
     * 
     *<span class="hljs-javadoctag" style="color:#66066;box-sizing: border-box;"> @return</span> 按照命令的顺序返回所有的数据
     */</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> List<Object> <span class="hljs-title" style="box-sizing: border-box;">syncAndReturnAll</span>() {
        List<Object> responseList = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> ArrayList<Object>();

        innerSync(responseList);

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> responseList;
    }

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">innerSync</span>(List<Object> formatted) {
        HashSet<Client> clientSet = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> HashSet<Client>();

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">try</span> {
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">for</span> (Client client : clients) {
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 在sync()调用时其实是不需要解析结果数据的,但是如果不调用get方法,</span>
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 发生了JedisMovedDataException这样的错误应用是不知道的,</span>
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 因此需要调用get()来触发错误。</span>
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 其实如果Response的data属性可以直接获取,可以省掉解析数据的时间,</span>
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 然而它并没有提供对应方法,要获取data属性就得用反射,</span>
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 不想再反射了,所以就这样了</span>
                Object data = generateResponse(client.getOne()).get();
                <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">null</span> != formatted) {
                    formatted.add(data);
                }

                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// size相同说明所有的client都已经添加,就不用再调用add方法了</span>
                <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (clientSet.size() != jedisMap.size()) {
                    clientSet.add(client);
                }
            }
        } <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">catch</span> (JedisRedirectionException jre) {
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (jre <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">instanceof</span> JedisMovedDataException) {
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// if MOVED redirection occurred, rebuilds cluster's slot cache,</span>
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// recommended by Redis cluster specification</span>
                refreshCluster();
            }

            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">throw</span> jre;
        } <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">finally</span> {
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (clientSet.size() != jedisMap.size()) {
                <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 所有还没有执行过的client要保证执行(flush),防止放回连接池后后面的命令被污染</span>
                <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">for</span> (Jedis jedis : jedisMap.values()) {
                    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (clientSet.contains(jedis.getClient())) {
                        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">continue</span>;
                    }

                    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">try</span> {
                        jedis.getClient().getAll();
                    } <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">catch</span> (RuntimeException ex) {
                        <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 其中一个client出问题,后面出问题的几率较大</span>
                    }
                }
            }

            close();
        }
    }

    <span class="hljs-annotation" style="color:#9b859d;box-sizing: border-box;">@Override</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">close</span>() {
        clean();

        clients.clear();

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">for</span> (Jedis jedis : jedisMap.values()) {
            jedis.close();
        }

        jedisMap.clear();
    }

    <span class="hljs-annotation" style="color:#9b859d;box-sizing: border-box;">@Override</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">protected</span> Client <span class="hljs-title" style="box-sizing: border-box;">getClient</span>(String key) {
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">byte</span>[] bKey = SafeEncoder.encode(key);

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> getClient(bKey);
    }

    <span class="hljs-annotation" style="color:#9b859d;box-sizing: border-box;">@Override</span>
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">protected</span> Client <span class="hljs-title" style="box-sizing: border-box;">getClient</span>(<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">byte</span>[] key) {
        Jedis jedis = getJedis(JedisClusterCRC16.getSlot(key));

        Client client = jedis.getClient();
        clients.add(client);

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> client;
    }

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> Jedis <span class="hljs-title" style="box-sizing: border-box;">getJedis</span>(<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">int</span> slot) {
        JedisClusterInfoCache cache = getValue(connectionHandler, FIELD_CACHE);

        JedisPool pool = cache.getSlotPool(slot);

        <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 根据pool从缓存中获取Jedis</span>
        Jedis jedis = jedisMap.get(pool);
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">if</span> (<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">null</span> == jedis) {
            jedis = pool.getResource();
            jedisMap.put(pool, jedis);
        }

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> jedis;
    }

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> Field <span class="hljs-title" style="box-sizing: border-box;">getField</span>(Class<?> cls, String fieldName) {
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">try</span> {
            Field field = cls.getDeclaredField(fieldName);
            field.setAccessible(<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">true</span>);

            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> field;
        } <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">catch</span> (NoSuchFieldException | SecurityException e) {
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">throw</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> RuntimeException(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"cannot find or access field '"</span> + fieldName + <span class="hljs-string" style="color:#0880;box-sizing: border-box;">"' from "</span> + cls.getName(), e);
        }
    }

    <span class="hljs-annotation" style="color:#9b859d;box-sizing: border-box;">@SuppressWarnings</span>({<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"unchecked"</span> })
    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> <T> T <span class="hljs-title" style="box-sizing: border-box;">getValue</span>(Object obj, Field field) {
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">try</span> {
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">return</span> (T)field.get(obj);
        } <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">catch</span> (IllegalArgumentException | IllegalAccessException e) {
            LOGGER.error(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"get value fail"</span>, e);

            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">throw</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> RuntimeException(e);
        }
    }

    <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">main</span>(String[] args) <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">throws</span> IOException {
        Set<HostAndPort> nodes = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> HashSet<HostAndPort>();
        nodes.add(<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> HostAndPort(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"127.0.0.1"</span>, <span class="hljs-number" style="color:#06666;box-sizing: border-box;">9379</span>));
        nodes.add(<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> HostAndPort(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"127.0.0.1"</span>, <span class="hljs-number" style="color:#06666;box-sizing: border-box;">9380</span>));

        JedisCluster jc = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">new</span> JedisCluster(nodes);

        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">long</span> s = System.currentTimeMillis();

        JedisClusterPipeline jcp = JedisClusterPipeline.pipelined(jc);
        jcp.refreshCluster();
        List<Object> batchResult = <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">null</span>;
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">try</span> {
            <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// batch write</span>
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">int</span> i = <span class="hljs-number" style="color:#06666;box-sizing: border-box;">0</span>; i < <span class="hljs-number" style="color:#06666;box-sizing: border-box;">10000</span>; i++) {
                jcp.set(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"k"</span> + i, <span class="hljs-string" style="color:#0880;box-sizing: border-box;">"v1"</span> + i);
            }
            jcp.sync();

            <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// batch read</span>
            <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">int</span> i = <span class="hljs-number" style="color:#06666;box-sizing: border-box;">0</span>; i < <span class="hljs-number" style="color:#06666;box-sizing: border-box;">10000</span>; i++) {
                jcp.get(<span class="hljs-string" style="color:#0880;box-sizing: border-box;">"k"</span> + i);
            }
            batchResult = jcp.syncAndReturnAll();
        } <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">finally</span> {
            jcp.close();
        }

        <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// output time </span>
        <span class="hljs-keyword" style="color:#0088;box-sizing: border-box;">long</span> t = System.currentTimeMillis() - s;
        System.out.println(t);

        System.out.println(batchResult.size());

        <span class="hljs-comment" style="color:#8800;box-sizing: border-box;">// 实际业务代码中,close要在finally中调,这里之所以没这么写,是因为懒</span>
        jc.close();
    }
}
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li><li style="box-sizing: border-box; padding: 0px 5px;">106</li><li style="box-sizing: border-box; padding: 0px 5px;">107</li><li style="box-sizing: border-box; padding: 0px 5px;">108</li><li style="box-sizing: border-box; padding: 0px 5px;">109</li><li style="box-sizing: border-box; padding: 0px 5px;">110</li><li style="box-sizing: border-box; padding: 0px 5px;">111</li><li style="box-sizing: border-box; padding: 0px 5px;">112</li><li style="box-sizing: border-box; padding: 0px 5px;">113</li><li style="box-sizing: border-box; padding: 0px 5px;">114</li><li style="box-sizing: border-box; padding: 0px 5px;">115</li><li style="box-sizing: border-box; padding: 0px 5px;">116</li><li style="box-sizing: border-box; padding: 0px 5px;">117</li><li style="box-sizing: border-box; padding: 0px 5px;">118</li><li style="box-sizing: border-box; padding: 0px 5px;">119</li><li style="box-sizing: border-box; padding: 0px 5px;">120</li><li style="box-sizing: border-box; padding: 0px 5px;">121</li><li style="box-sizing: border-box; padding: 0px 5px;">122</li><li style="box-sizing: border-box; padding: 0px 5px;">123</li><li style="box-sizing: border-box; padding: 0px 5px;">124</li><li style="box-sizing: border-box; padding: 0px 5px;">125</li><li style="box-sizing: border-box; padding: 0px 5px;">126</li><li style="box-sizing: border-box; padding: 0px 5px;">127</li><li style="box-sizing: border-box; padding: 0px 5px;">128</li><li style="box-sizing: border-box; padding: 0px 5px;">129</li><li style="box-sizing: border-box; padding: 0px 5px;">130</li><li style="box-sizing: border-box; padding: 0px 5px;">131</li><li style="box-sizing: border-box; padding: 0px 5px;">132</li><li style="box-sizing: border-box; padding: 0px 5px;">133</li><li style="box-sizing: border-box; padding: 0px 5px;">134</li><li style="box-sizing: border-box; padding: 0px 5px;">135</li><li style="box-sizing: border-box; padding: 0px 5px;">136</li><li style="box-sizing: border-box; padding: 0px 5px;">137</li><li style="box-sizing: border-box; padding: 0px 5px;">138</li><li style="box-sizing: border-box; padding: 0px 5px;">139</li><li style="box-sizing: border-box; padding: 0px 5px;">140</li><li style="box-sizing: border-box; padding: 0px 5px;">141</li><li style="box-sizing: border-box; padding: 0px 5px;">142</li><li style="box-sizing: border-box; padding: 0px 5px;">143</li><li style="box-sizing: border-box; padding: 0px 5px;">144</li><li style="box-sizing: border-box; padding: 0px 5px;">145</li><li style="box-sizing: border-box; padding: 0px 5px;">146</li><li style="box-sizing: border-box; padding: 0px 5px;">147</li><li style="box-sizing: border-box; padding: 0px 5px;">148</li><li style="box-sizing: border-box; padding: 0px 5px;">149</li><li style="box-sizing: border-box; padding: 0px 5px;">150</li><li style="box-sizing: border-box; padding: 0px 5px;">151</li><li style="box-sizing: border-box; padding: 0px 5px;">152</li><li style="box-sizing: border-box; padding: 0px 5px;">153</li><li style="box-sizing: border-box; padding: 0px 5px;">154</li><li style="box-sizing: border-box; padding: 0px 5px;">155</li><li style="box-sizing: border-box; padding: 0px 5px;">156</li><li style="box-sizing: border-box; padding: 0px 5px;">157</li><li style="box-sizing: border-box; padding: 0px 5px;">158</li><li style="box-sizing: border-box; padding: 0px 5px;">159</li><li style="box-sizing: border-box; padding: 0px 5px;">160</li><li style="box-sizing: border-box; padding: 0px 5px;">161</li><li style="box-sizing: border-box; padding: 0px 5px;">162</li><li style="box-sizing: border-box; padding: 0px 5px;">163</li><li style="box-sizing: border-box; padding: 0px 5px;">164</li><li style="box-sizing: border-box; padding: 0px 5px;">165</li><li style="box-sizing: border-box; padding: 0px 5px;">166</li><li style="box-sizing: border-box; padding: 0px 5px;">167</li><li style="box-sizing: border-box; padding: 0px 5px;">168</li><li style="box-sizing: border-box; padding: 0px 5px;">169</li><li style="box-sizing: border-box; padding: 0px 5px;">170</li><li style="box-sizing: border-box; padding: 0px 5px;">171</li><li style="box-sizing: border-box; padding: 0px 5px;">172</li><li style="box-sizing: border-box; padding: 0px 5px;">173</li><li style="box-sizing: border-box; padding: 0px 5px;">174</li><li style="box-sizing: border-box; padding: 0px 5px;">175</li><li style="box-sizing: border-box; padding: 0px 5px;">176</li><li style="box-sizing: border-box; padding: 0px 5px;">177</li><li style="box-sizing: border-box; padding: 0px 5px;">178</li><li style="box-sizing: border-box; padding: 0px 5px;">179</li><li style="box-sizing: border-box; padding: 0px 5px;">180</li><li style="box-sizing: border-box; padding: 0px 5px;">181</li><li style="box-sizing: border-box; padding: 0px 5px;">182</li><li style="box-sizing: border-box; padding: 0px 5px;">183</li><li style="box-sizing: border-box; padding: 0px 5px;">184</li><li style="box-sizing: border-box; padding: 0px 5px;">185</li><li style="box-sizing: border-box; padding: 0px 5px;">186</li><li style="box-sizing: border-box; padding: 0px 5px;">187</li><li style="box-sizing: border-box; padding: 0px 5px;">188</li><li style="box-sizing: border-box; padding: 0px 5px;">189</li><li style="box-sizing: border-box; padding: 0px 5px;">190</li><li style="box-sizing: border-box; padding: 0px 5px;">191</li><li style="box-sizing: border-box; padding: 0px 5px;">192</li><li style="box-sizing: border-box; padding: 0px 5px;">193</li><li style="box-sizing: border-box; padding: 0px 5px;">194</li><li style="box-sizing: border-box; padding: 0px 5px;">195</li><li style="box-sizing: border-box; padding: 0px 5px;">196</li><li style="box-sizing: border-box; padding: 0px 5px;">197</li><li style="box-sizing: border-box; padding: 0px 5px;">198</li><li style="box-sizing: border-box; padding: 0px 5px;">199</li><li style="box-sizing: border-box; padding: 0px 5px;">200</li><li style="box-sizing: border-box; padding: 0px 5px;">201</li><li style="box-sizing: border-box; padding: 0px 5px;">202</li><li style="box-sizing: border-box; padding: 0px 5px;">203</li><li style="box-sizing: border-box; padding: 0px 5px;">204</li><li style="box-sizing: border-box; padding: 0px 5px;">205</li><li style="box-sizing: border-box; padding: 0px 5px;">206</li><li style="box-sizing: border-box; padding: 0px 5px;">207</li><li style="box-sizing: border-box; padding: 0px 5px;">208</li><li style="box-sizing: border-box; padding: 0px 5px;">209</li><li style="box-sizing: border-box; padding: 0px 5px;">210</li><li style="box-sizing: border-box; padding: 0px 5px;">211</li><li style="box-sizing: border-box; padding: 0px 5px;">212</li><li style="box-sizing: border-box; padding: 0px 5px;">213</li><li style="box-sizing: border-box; padding: 0px 5px;">214</li><li style="box-sizing: border-box; padding: 0px 5px;">215</li><li style="box-sizing: border-box; padding: 0px 5px;">216</li><li style="box-sizing: border-box; padding: 0px 5px;">217</li><li style="box-sizing: border-box; padding: 0px 5px;">218</li><li style="box-sizing: border-box; padding: 0px 5px;">219</li><li style="box-sizing: border-box; padding: 0px 5px;">220</li><li style="box-sizing: border-box; padding: 0px 5px;">221</li><li style="box-sizing: border-box; padding: 0px 5px;">222</li><li style="box-sizing: border-box; padding: 0px 5px;">223</li><li style="box-sizing: border-box; padding: 0px 5px;">224</li><li style="box-sizing: border-box; padding: 0px 5px;">225</li><li style="box-sizing: border-box; padding: 0px 5px;">226</li><li style="box-sizing: border-box; padding: 0px 5px;">227</li><li style="box-sizing: border-box; padding: 0px 5px;">228</li><li style="box-sizing: border-box; padding: 0px 5px;">229</li><li style="box-sizing: border-box; padding: 0px 5px;">230</li><li style="box-sizing: border-box; padding: 0px 5px;">231</li><li style="box-sizing: border-box; padding: 0px 5px;">232</li><li style="box-sizing: border-box; padding: 0px 5px;">233</li><li style="box-sizing: border-box; padding: 0px 5px;">234</li><li style="box-sizing: border-box; padding: 0px 5px;">235</li><li style="box-sizing: border-box; padding: 0px 5px;">236</li><li style="box-sizing: border-box; padding: 0px 5px;">237</li><li style="box-sizing: border-box; padding: 0px 5px;">238</li><li style="box-sizing: border-box; padding: 0px 5px;">239</li><li style="box-sizing: border-box; padding: 0px 5px;">240</li><li style="box-sizing: border-box; padding: 0px 5px;">241</li><li style="box-sizing: border-box; padding: 0px 5px;">242</li><li style="box-sizing: border-box; padding: 0px 5px;">243</li><li style="box-sizing: border-box; padding: 0px 5px;">244</li><li style="box-sizing: border-box; padding: 0px 5px;">245</li><li style="box-sizing: border-box; padding: 0px 5px;">246</li><li style="box-sizing: border-box; padding: 0px 5px;">247</li><li style="box-sizing: border-box; padding: 0px 5px;">248</li><li style="box-sizing: border-box; padding: 0px 5px;">249</li><li style="box-sizing: border-box; padding: 0px 5px;">250</li><li style="box-sizing: border-box; padding: 0px 5px;">251</li><li style="box-sizing: border-box; padding: 0px 5px;">252</li><li style="box-sizing: border-box; padding: 0px 5px;">253</li><li style="box-sizing: border-box; padding: 0px 5px;">254</li><li style="box-sizing: border-box; padding: 0px 5px;">255</li><li style="box-sizing: border-box; padding: 0px 5px;">256</li><li style="box-sizing: border-box; padding: 0px 5px;">257</li><li style="box-sizing: border-box; padding: 0px 5px;">258</li><li style="box-sizing: border-box; padding: 0px 5px;">259</li><li style="box-sizing: border-box; padding: 0px 5px;">260</li><li style="box-sizing: border-box; padding: 0px 5px;">261</li><li style="box-sizing: border-box; padding: 0px 5px;">262</li><li style="box-sizing: border-box; padding: 0px 5px;">263</li><li style="box-sizing: border-box; padding: 0px 5px;">264</li><li style="box-sizing: border-box; padding: 0px 5px;">265</li><li style="box-sizing: border-box; padding: 0px 5px;">266</li><li style="box-sizing: border-box; padding: 0px 5px;">267</li><li style="box-sizing: border-box; padding: 0px 5px;">268</li><li style="box-sizing: border-box; padding: 0px 5px;">269</li><li style="box-sizing: border-box; padding: 0px 5px;">270</li><li style="box-sizing: border-box; padding: 0px 5px;">271</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li><li style="box-sizing: border-box; padding: 0px 5px;">106</li><li style="box-sizing: border-box; padding: 0px 5px;">107</li><li style="box-sizing: border-box; padding: 0px 5px;">108</li><li style="box-sizing: border-box; padding: 0px 5px;">109</li><li style="box-sizing: border-box; padding: 0px 5px;">110</li><li style="box-sizing: border-box; padding: 0px 5px;">111</li><li style="box-sizing: border-box; padding: 0px 5px;">112</li><li style="box-sizing: border-box; padding: 0px 5px;">113</li><li style="box-sizing: border-box; padding: 0px 5px;">114</li><li style="box-sizing: border-box; padding: 0px 5px;">115</li><li style="box-sizing: border-box; padding: 0px 5px;">116</li><li style="box-sizing: border-box; padding: 0px 5px;">117</li><li style="box-sizing: border-box; padding: 0px 5px;">118</li><li style="box-sizing: border-box; padding: 0px 5px;">119</li><li style="box-sizing: border-box; padding: 0px 5px;">120</li><li style="box-sizing: border-box; padding: 0px 5px;">121</li><li style="box-sizing: border-box; padding: 0px 5px;">122</li><li style="box-sizing: border-box; padding: 0px 5px;">123</li><li style="box-sizing: border-box; padding: 0px 5px;">124</li><li style="box-sizing: border-box; padding: 0px 5px;">125</li><li style="box-sizing: border-box; padding: 0px 5px;">126</li><li style="box-sizing: border-box; padding: 0px 5px;">127</li><li style="box-sizing: border-box; padding: 0px 5px;">128</li><li style="box-sizing: border-box; padding: 0px 5px;">129</li><li style="box-sizing: border-box; padding: 0px 5px;">130</li><li style="box-sizing: border-box; padding: 0px 5px;">131</li><li style="box-sizing: border-box; padding: 0px 5px;">132</li><li style="box-sizing: border-box; padding: 0px 5px;">133</li><li style="box-sizing: border-box; padding: 0px 5px;">134</li><li style="box-sizing: border-box; padding: 0px 5px;">135</li><li style="box-sizing: border-box; padding: 0px 5px;">136</li><li style="box-sizing: border-box; padding: 0px 5px;">137</li><li style="box-sizing: border-box; padding: 0px 5px;">138</li><li style="box-sizing: border-box; padding: 0px 5px;">139</li><li style="box-sizing: border-box; padding: 0px 5px;">140</li><li style="box-sizing: border-box; padding: 0px 5px;">141</li><li style="box-sizing: border-box; padding: 0px 5px;">142</li><li style="box-sizing: border-box; padding: 0px 5px;">143</li><li style="box-sizing: border-box; padding: 0px 5px;">144</li><li style="box-sizing: border-box; padding: 0px 5px;">145</li><li style="box-sizing: border-box; padding: 0px 5px;">146</li><li style="box-sizing: border-box; padding: 0px 5px;">147</li><li style="box-sizing: border-box; padding: 0px 5px;">148</li><li style="box-sizing: border-box; padding: 0px 5px;">149</li><li style="box-sizing: border-box; padding: 0px 5px;">150</li><li style="box-sizing: border-box; padding: 0px 5px;">151</li><li style="box-sizing: border-box; padding: 0px 5px;">152</li><li style="box-sizing: border-box; padding: 0px 5px;">153</li><li style="box-sizing: border-box; padding: 0px 5px;">154</li><li style="box-sizing: border-box; padding: 0px 5px;">155</li><li style="box-sizing: border-box; padding: 0px 5px;">156</li><li style="box-sizing: border-box; padding: 0px 5px;">157</li><li style="box-sizing: border-box; padding: 0px 5px;">158</li><li style="box-sizing: border-box; padding: 0px 5px;">159</li><li style="box-sizing: border-box; padding: 0px 5px;">160</li><li style="box-sizing: border-box; padding: 0px 5px;">161</li><li style="box-sizing: border-box; padding: 0px 5px;">162</li><li style="box-sizing: border-box; padding: 0px 5px;">163</li><li style="box-sizing: border-box; padding: 0px 5px;">164</li><li style="box-sizing: border-box; padding: 0px 5px;">165</li><li style="box-sizing: border-box; padding: 0px 5px;">166</li><li style="box-sizing: border-box; padding: 0px 5px;">167</li><li style="box-sizing: border-box; padding: 0px 5px;">168</li><li style="box-sizing: border-box; padding: 0px 5px;">169</li><li style="box-sizing: border-box; padding: 0px 5px;">170</li><li style="box-sizing: border-box; padding: 0px 5px;">171</li><li style="box-sizing: border-box; padding: 0px 5px;">172</li><li style="box-sizing: border-box; padding: 0px 5px;">173</li><li style="box-sizing: border-box; padding: 0px 5px;">174</li><li style="box-sizing: border-box; padding: 0px 5px;">175</li><li style="box-sizing: border-box; padding: 0px 5px;">176</li><li style="box-sizing: border-box; padding: 0px 5px;">177</li><li style="box-sizing: border-box; padding: 0px 5px;">178</li><li style="box-sizing: border-box; padding: 0px 5px;">179</li><li style="box-sizing: border-box; padding: 0px 5px;">180</li><li style="box-sizing: border-box; padding: 0px 5px;">181</li><li style="box-sizing: border-box; padding: 0px 5px;">182</li><li style="box-sizing: border-box; padding: 0px 5px;">183</li><li style="box-sizing: border-box; padding: 0px 5px;">184</li><li style="box-sizing: border-box; padding: 0px 5px;">185</li><li style="box-sizing: border-box; padding: 0px 5px;">186</li><li style="box-sizing: border-box; padding: 0px 5px;">187</li><li style="box-sizing: border-box; padding: 0px 5px;">188</li><li style="box-sizing: border-box; padding: 0px 5px;">189</li><li style="box-sizing: border-box; padding: 0px 5px;">190</li><li style="box-sizing: border-box; padding: 0px 5px;">191</li><li style="box-sizing: border-box; padding: 0px 5px;">192</li><li style="box-sizing: border-box; padding: 0px 5px;">193</li><li style="box-sizing: border-box; padding: 0px 5px;">194</li><li style="box-sizing: border-box; padding: 0px 5px;">195</li><li style="box-sizing: border-box; padding: 0px 5px;">196</li><li style="box-sizing: border-box; padding: 0px 5px;">197</li><li style="box-sizing: border-box; padding: 0px 5px;">198</li><li style="box-sizing: border-box; padding: 0px 5px;">199</li><li style="box-sizing: border-box; padding: 0px 5px;">200</li><li style="box-sizing: border-box; padding: 0px 5px;">201</li><li style="box-sizing: border-box; padding: 0px 5px;">202</li><li style="box-sizing: border-box; padding: 0px 5px;">203</li><li style="box-sizing: border-box; padding: 0px 5px;">204</li><li style="box-sizing: border-box; padding: 0px 5px;">205</li><li style="box-sizing: border-box; padding: 0px 5px;">206</li><li style="box-sizing: border-box; padding: 0px 5px;">207</li><li style="box-sizing: border-box; padding: 0px 5px;">208</li><li style="box-sizing: border-box; padding: 0px 5px;">209</li><li style="box-sizing: border-box; padding: 0px 5px;">210</li><li style="box-sizing: border-box; padding: 0px 5px;">211</li><li style="box-sizing: border-box; padding: 0px 5px;">212</li><li style="box-sizing: border-box; padding: 0px 5px;">213</li><li style="box-sizing: border-box; padding: 0px 5px;">214</li><li style="box-sizing: border-box; padding: 0px 5px;">215</li><li style="box-sizing: border-box; padding: 0px 5px;">216</li><li style="box-sizing: border-box; padding: 0px 5px;">217</li><li style="box-sizing: border-box; padding: 0px 5px;">218</li><li style="box-sizing: border-box; padding: 0px 5px;">219</li><li style="box-sizing: border-box; padding: 0px 5px;">220</li><li style="box-sizing: border-box; padding: 0px 5px;">221</li><li style="box-sizing: border-box; padding: 0px 5px;">222</li><li style="box-sizing: border-box; padding: 0px 5px;">223</li><li style="box-sizing: border-box; padding: 0px 5px;">224</li><li style="box-sizing: border-box; padding: 0px 5px;">225</li><li style="box-sizing: border-box; padding: 0px 5px;">226</li><li style="box-sizing: border-box; padding: 0px 5px;">227</li><li style="box-sizing: border-box; padding: 0px 5px;">228</li><li style="box-sizing: border-box; padding: 0px 5px;">229</li><li style="box-sizing: border-box; padding: 0px 5px;">230</li><li style="box-sizing: border-box; padding: 0px 5px;">231</li><li style="box-sizing: border-box; padding: 0px 5px;">232</li><li style="box-sizing: border-box; padding: 0px 5px;">233</li><li style="box-sizing: border-box; padding: 0px 5px;">234</li><li style="box-sizing: border-box; padding: 0px 5px;">235</li><li style="box-sizing: border-box; padding: 0px 5px;">236</li><li style="box-sizing: border-box; padding: 0px 5px;">237</li><li style="box-sizing: border-box; padding: 0px 5px;">238</li><li style="box-sizing: border-box; padding: 0px 5px;">239</li><li style="box-sizing: border-box; padding: 0px 5px;">240</li><li style="box-sizing: border-box; padding: 0px 5px;">241</li><li style="box-sizing: border-box; padding: 0px 5px;">242</li><li style="box-sizing: border-box; padding: 0px 5px;">243</li><li style="box-sizing: border-box; padding: 0px 5px;">244</li><li style="box-sizing: border-box; padding: 0px 5px;">245</li><li style="box-sizing: border-box; padding: 0px 5px;">246</li><li style="box-sizing: border-box; padding: 0px 5px;">247</li><li style="box-sizing: border-box; padding: 0px 5px;">248</li><li style="box-sizing: border-box; padding: 0px 5px;">249</li><li style="box-sizing: border-box; padding: 0px 5px;">250</li><li style="box-sizing: border-box; padding: 0px 5px;">251</li><li style="box-sizing: border-box; padding: 0px 5px;">252</li><li style="box-sizing: border-box; padding: 0px 5px;">253</li><li style="box-sizing: border-box; padding: 0px 5px;">254</li><li style="box-sizing: border-box; padding: 0px 5px;">255</li><li style="box-sizing: border-box; padding: 0px 5px;">256</li><li style="box-sizing: border-box; padding: 0px 5px;">257</li><li style="box-sizing: border-box; padding: 0px 5px;">258</li><li style="box-sizing: border-box; padding: 0px 5px;">259</li><li style="box-sizing: border-box; padding: 0px 5px;">260</li><li style="box-sizing: border-box; padding: 0px 5px;">261</li><li style="box-sizing: border-box; padding: 0px 5px;">262</li><li style="box-sizing: border-box; padding: 0px 5px;">263</li><li style="box-sizing: border-box; padding: 0px 5px;">264</li><li style="box-sizing: border-box; padding: 0px 5px;">265</li><li style="box-sizing: border-box; padding: 0px 5px;">266</li><li style="box-sizing: border-box; padding: 0px 5px;">267</li><li style="box-sizing: border-box; padding: 0px 5px;">268</li><li style="box-sizing: border-box; padding: 0px 5px;">269</li><li style="box-sizing: border-box; padding: 0px 5px;">270</li><li style="box-sizing: border-box; padding: 0px 5px;">271</li></ul>


FROM:  http://blog.csdn.net/youaremoon/article/details/51751991


  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值