之前nosql群里的哥们对redis讨论的挺热闹,今天我也着手试了吧,主要测试2个方面(只是简单测试了下,主要针对jedis java客户端,jedis已实现了consistent hash算法)
1:在多节点下,数据的分布均匀情况如何
2:在多节点下,删除可增加节点后,对原有数据的映射关系,及命中情况如何
本机上启动3个节点(6379、6380、6381)
第一:测试数据分布测试情况:
3个节点下,插入10000条数据情况,数据分布还是挺理想的
节点 | 数据量 | 分布率 |
6379 node | 3365 | 33.65% |
6380 node | 3243 | 32.43% |
6381 node | 3392 | 33.92% |
第二:测试增删节点后,对原有数据的影响
a:首先本机的3个节点(6379、6380、6381),插入10条数据(数据少,分布的还是不均匀的),数据的分布情况为:
key:0 put in :127.0.0.1:6380
key:1 put in :127.0.0.1:6380
key:2 put in :127.0.0.1:6381
key:3 put in :127.0.0.1:6381
key:4 put in :127.0.0.1:6380
key:5 put in :127.0.0.1:6381
key:6 put in :127.0.0.1:6379
key:7 put in :127.0.0.1:6380
key:8 put in :127.0.0.1:6381
key:9 put in :127.0.0.1:6379
b:再把6381节点服务停掉,代码里面不修改节点配置,发现该正确hit的key还是能正确hit到自己的节点(此时后台会抛连接redis异常)
key:0 from :127.0.0.1:6380->0
key:1 from :127.0.0.1:6380->1
key:2 from :127.0.0.1:6381->null
key:3 from :127.0.0.1:6381->null
key:4 from :127.0.0.1:6380->4
key:5 from :127.0.0.1:6381->null
key:6 from :127.0.0.1:6379->6
key:7 from :127.0.0.1:6380->7
key:8 from :127.0.0.1:6381->null
key:9 from :127.0.0.1:6379->9
c:再启动6381这个节点服务,代码里面的配置remove掉6381,然后重新get所有的key,可以看到该正确hit的key也还是能正确hit到自己的节点
key:0 from :127.0.0.1:6380->0
key:1 from :127.0.0.1:6380->1
key:2 from :127.0.0.1:6380->null
key:3 from :127.0.0.1:6380->null
key:4 from :127.0.0.1:6380->4
key:5 from :127.0.0.1:6380->null
key:6 from :127.0.0.1:6379->6
key:7 from :127.0.0.1:6380->7
key:8 from :127.0.0.1:6380->null
key:9 from :127.0.0.1:6379->9
其实以上测试已经说明了,jedis客服端在移除或者添加一个cache节点时,不影响原有数据的映射关系,这也是一致性hash算法的原理。就这么测这么点,自己玩玩而已,至于性能如何,官方说的挺不错的,大家可以自己去看看...整一个新的分布式缓存我觉得这些必要的测试还是需要的,下面把测试代码发出来,写的很简单,就2个类:
import java.util.ArrayList;
import java.util.List;
import redis.clients.jedis.JedisPoolConfig;
import redis.clients.jedis.JedisShardInfo;
import redis.clients.jedis.ShardedJedis;
import redis.clients.jedis.ShardedJedisPool;
import redis.clients.util.Hashing;
import redis.clients.util.Sharded;
/**
*
* @Title: RedisResourcePool.java
* @Copyright: Copyright (c) 2005
* @Description: <br> <br>
* @Company: hd
* @Created on May 4, 2012 3:05:44 PM
* @author <a href="mailTo:tangzhibin@*.com">tangzhibin</a>
*/
public class RedisResourcePool {
/**ShardedJedisPool底层实现了commons.pool中的GenericObjectPool对象池,it's thread-safety*/
private ShardedJedisPool shareRedisPool;
void initRedisPool(){
JedisPoolConfig configs=new JedisPoolConfig();
/**对象池中最大活动的对象个数*/
configs.setMaxActive(200);
/**对象的最大空闲时间*/
configs.setMaxIdle(60*1000);
/**客户端获取对象时,如果池中没有空闲的对象(已达MaxActive),则会等待MaxWait时间,在此时间内,如果还没有空闲的对象返回,则抛出异常*/
configs.setMaxWait(5*1000);
/**设置host port等配置*/
String []hostPorts={"127.0.0.1:6379","127.0.0.1:6380","127.0.0.1:6381"};
//String []hostPorts={"127.0.0.1:6379","127.0.0.1:6380"};
List<JedisShardInfo> jdsInfoList =new ArrayList<JedisShardInfo>(hostPorts.length);
for(int i=0;i<hostPorts.length;i++){
String hostPort=hostPorts[i];
String[] host_port=hostPort.split(":");
String host=host_port[0];
/**注意port一定要转为int类型(不能为String),否则JedisShardInfod初始化时, 会将该String作为节点name名称,而把默认的6379作为端口号*/
int port=Integer.parseInt(host_port[1]);
JedisShardInfo jedisShardInfo=new JedisShardInfo(host,port);
jdsInfoList.add(jedisShardInfo);
}
/**初始化redis池*/
this.shareRedisPool = new ShardedJedisPool(configs, jdsInfoList,
Hashing.MURMUR_HASH, Sharded.DEFAULT_KEY_TAG_PATTERN);
}
private RedisResourcePool(){
initRedisPool();
}
static class RedisResourcePoolIniter{
private static final RedisResourcePool redisResourcePool=new RedisResourcePool();
private static RedisResourcePool getRedisResourcePool(){
return redisResourcePool;
}
}
public ShardedJedis getRedisCacheResource(){
return shareRedisPool.getResource();
}
public void restoreRedisCacheResource(ShardedJedis resource){
if(shareRedisPool!=null && resource!=null){
shareRedisPool.returnResource(resource);
}
}
public static RedisResourcePool getRedisResourcePool(){
return RedisResourcePoolIniter.getRedisResourcePool();
}
}
测试的main函数在此类中
import redis.clients.jedis.Client;
import redis.clients.jedis.ShardedJedis;
/**
*
* @Title: RedisManager.java
* @Copyright: Copyright (c) 2005
* @Description: <br> <br>
* @Company: hd
* @Created on May 4, 2012 3:05:44 PM
* @author <a href="mailTo:tangzhibin">tangzhibin</a>
*/
public class RedisManager {
private RedisResourcePool redisResourcePool=RedisResourcePool.getRedisResourcePool();
private RedisManager(){}
private static class RedisManagerIniter{
private static final RedisManager redisManager=new RedisManager();
private static RedisManager getRedisManager(){
return redisManager;
}
}
public static RedisManager getRedisManager(){
return RedisManagerIniter.getRedisManager();
}
public String set(String key,String value){
ShardedJedis resource=null;
try{
resource=redisResourcePool.getRedisCacheResource();
Client cli=resource.getShard(key).getClient();
return resource.set(key, value);
}finally{
redisResourcePool.restoreRedisCacheResource(resource);
}
}
public Object get(String key){
ShardedJedis resource=null;
try{
resource=redisResourcePool.getRedisCacheResource();
Client cli=resource.getShard(key).getClient();
System.out.println("key:"+key+" from :"+cli.getHost()+":"+cli.getPort());
return resource.get(key);
}catch(Exception ex){
ex.printStackTrace();
return null;
}finally{
redisResourcePool.restoreRedisCacheResource(resource);
}
}
public static void main(String[] args){
RedisManager rm=RedisManager.getRedisManager();
//set
for(int i=0;i<10000;i++){
rm.set(String.valueOf(i),String.valueOf(i));
}
// //get
// for(int i=0;i<10;i++){
// System.out.println(rm.get(String.valueOf(i)));
// }
//
}
}