基于nutcracker 的redis分布式缓存的实现的注意

原文链接：http://blog.csdn.net/wenbingoon/article/details/24482001

参考链接：http://os.51cto.com/art/201408/447247_all.htm

启动参数

Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file]
                  [-c conf file] [-s stats port] [-a stats addr]
                  [-i stats interval] [-p pid file] [-m mbuf size]

Options:
  -h, --help             : this help
  -V, --version          : show version and exit
  -t, --test-conf        : 测试配置文件是否存在或者是否存在语法错误
  -d, --daemonize        : 守护进程启动
  -D, --describe-stats   : 打印状态
  -v, --verbosity=N      : 设置日志级别 (default: 5, min: 0, max: 11)
  -o, --output=S         : 设置日志文件位置 (default: stderr)
  -c, --conf-file=S      : 设置配置文件位置 (default: conf/nutcracker.yml)
  -s, --stats-port=N     : 设置状态监控端口 (default: 22222)
  -a, --stats-addr=S     : 设置状态监控IP (default: 0.0.0.0)
  -i, --stats-interval=N : 设置状态监控聚合时间间隔 (default: 30000 msec)
  -p, --pid-file=S       : 设置pid文件 (default: off)
  -m, --mbuf-size=N      : 设置缓存块的大小 (default: 16384 bytes)

一般我会这样启动：

 nutcracker -d -c /usr/local/nutcracker/nutcracker.yml -o /usr/local/nutcracker/nutcracker.log

支持的命令

string command:
   set,get,getset,mget,getrange,incr,decr,incrby,incrbyfloat,decrby,setbit,setex,setnx,setrange,strlen, psetex
keys command:  
    del,dump,exists,expire,expireat,persist,pexpire,pttl,restore
hashes command:
    hdel,hexists,hget,hgetall,hincrby,hincrbyfloat,hkeys,hlen,hmget,hmset,hset,hsetnx,hvals
list command:
    lindex,linsert,llen,lpop,lpush,lpushx,lrange,lrem,lset,limit,rpop,rpoplpush,rpush,rpushx  
set command:
    sadd,scard,sdiff,sdiffstore,sinter,sinterstore,sismember,
smembers,smove,spop,srandmember,serm,sunion,sunionstore
sorted sets command:
    zadd,zcard,zcount,zincrby,zincrby,zinterstore,zrange,zrangebyscore,zrank,zrem,zremrangebyrank,zrevrangebyscore,zrevrank,zscore,zunionstore
scripting command:
    eval,evalsha

暂不支持的命令

keys command:
     keys,migrate,move object,randomkey,rename,renamenx,sort
strings command:
     bitop,mset,msetnx
list command:
     blpop,brpop,brpoplpush
scripting command:
     script exists,script flush,script kill,script load
pub/sub command:(全部不支持)
     psubscribe,publish,punsubscribe,subscribe,unsubscribe
server command:(全部不支持)
     bgrewriteaof,bgsave,client kill,client list,config get,config set, config resetstat,dbsize,flushdb,flushall,lastsave,monitor,save,shutdown,slaveof
connection command:(全部不支持)
     auth,echo,ping,quit,select
transactions command:(全部不支持)
     discard,exec,multi,unwatch,watch

状态监控

twemproxy自带监控端口，默认端口为222222，可以在启动时候使用-s 选项指定。监控状态如下

[root@slave nutcracker]# nc 127.0.0.1 22222
{"service":"nutcracker", "source":"slave", "version":"0.2.4", "uptime":103, "timestamp":1380011450, 
 "redis1": {"client_eof":0, "client_err":0,   "client_connections":0, "server_ejects":0, "forward_error":0, "fragments":0, 
 "server1": {"server_eof":0, "server_err":0,     "server_timedout":0,"server_connections":0, "requests":0, "request_bytes":0, "responses":0, "response_bytes":0, "in_queue":0, "in_queue_bytes":0, "out_queue":0, "out_queue_bytes":0},
  "server2": {"server_eof":0, "server_err":0, "server_timedout":0, "server_connections":0, "requests":0, "request_bytes":0, "responses":0, "response_bytes":0, "in_queue":0, "in_queue_bytes":0, "out_queue":0, "out_queue_bytes":0},
  "server3": {"server_eof":0, "server_err":0, "server_timedout":0, "server_connections":0, "requests":0, "request_bytes":0, "responses":0, "response_bytes":0, "in_queue":0, "in_queue_bytes":0, "out_queue":0, "out_queue_bytes":0},
  "server4": {"server_eof":0, "server_err":0, "server_timedout":0, "server_connections":0, "requests":0, "request_bytes":0, "responses":0, "response_bytes":0, "in_queue":0, "in_queue_bytes":0, "out_queue":0, "out_queue_bytes":0}}}

可以看出，返回的信息主要分为三部分，twemproxy 概要信息，每个集群信息，每个集群下每个实例的信息。 可以使用 nutcracker -D 或者 -describe-stats 打印所有的状态描述信息

$ nutcracker --describe-stats

    pool stats:
        client_eof          "# eof on client connections"
        client_err          "# errors on client connections"
        client_connections  "# active client connections"
        server_ejects       "# times backend server was ejected"
        forward_error       "# times we encountered a forwarding error"
        fragments           "# fragments created from a multi-vector request"

    server stats:
        server_eof          "# eof on server connections"
        server_err          "# errors on server connections"
        server_timedout     "# timeouts on server connections"
        server_connections  "# active server connections"
        requests            "# requests"
        request_bytes       "total request bytes"
        responses           "# responses"
        response_bytes      "total response bytes"
        in_queue            "# requests in incoming queue"
        in_queue_bytes      "current request bytes in incoming queue"
        out_queue           "# requests in outgoing queue"
        out_queue_bytes     "current request bytes in outgoing queue"

高可用

不存在单点的问题：开启多个相同配置的twemproxy实例，保持客户端链接到一个可用的twemproxy上。每个twemproxy上可以配置多个代理集群。

性能

根据 Redis 作者的测试结果，在大多数情况下，Twemproxy 的性能相当不错，直接操作 Redis 相比，最多只有20%的性能损失。这对于它带来的好处来说真的是微不足道了。唯一可能还有待改进的是其 MGET 操作的效率，其性能只有直接操作 Redis 的 50%。

redis-benchmark against redis-server

$redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379
SET: 89285.71 requests per second
GET: 92592.59 requests per second
INCR: 89285.71 requests per second
LPUSH: 90090.09 requests per second
LPOP: 90090.09 requests per second
SADD: 90090.09 requests per second
SPOP: 93457.95 requests per second
LPUSH (needed to benchmark LRANGE): 89285.71 requests per second
LRANGE_100 (first 100 elements): 36496.35 requests per second
LRANGE_300 (first 300 elements): 15748.03 requests per second
LRANGE_500 (first 450 elements): 11135.86 requests per second
LRANGE_600 (first 600 elements): 8650.52 requests per second

redis-benchmark against nutcracker proxing redis-server

$ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 22121
SET: 85470.09 requests per second
GET: 86956.52 requests per second
INCR: 85470.09 requests per second
LPUSH: 84745.77 requests per second
LPOP: 86206.90 requests per second
SADD: 84745.77 requests per second
SPOP: 86956.52 requests per second
LPUSH (needed to benchmark LRANGE): 84745.77 requests per second
LRANGE_100 (first 100 elements): 29761.90 requests per second
LRANGE_300 (first 300 elements): 12376.24 requests per second
LRANGE_500 (first 450 elements): 8605.85 requests per second
LRANGE_600 (first 600 elements): 6587.62 requests per second 

问题和不足

• 不支持针对多个值的操作，比如取sets的子交并补等（MGET 和 DEL 除外）
• 不支持Redis的事务操作
• 出错提示还不够完善