一致性hash算法在Memcache Java client中的应用

最新推荐文章于 2018-03-07 19:24:21 发布

zhangbinalan

最新推荐文章于 2018-03-07 19:24:21 发布

阅读量512

点赞数

分类专栏： NOSQL

本文链接：https://blog.csdn.net/zhangbinalan/article/details/37907983

版权

NOSQL 专栏收录该内容

10 篇文章 0 订阅

订阅专栏

Memcache Java Client需要通过应用hash算法来决定内容存放到哪一台memcache服务器，学习了一下http://github.com/gwhalin/Memcached-Java-Client/的源代码，本文记录一下代码里面涉及到的hash算法；

关于一致性hash，可以看一下http://blog.csdn.net/zhangbinalan/article/details/37889853；

一：代码里面提到了四种hash方法

1:NATIVE_HASH

key.hashCode()

2:OLD_COMPAT_HASH

	private static long origCompatHashingAlg( String key ) {
		long hash   = 0;
		char[] cArr = key.toCharArray();
		for ( int i = 0; i < cArr.length; ++i ) {
			hash = (hash * 33) + cArr[i];
		}
		return hash;
	}

3:NEW_COMPAT_HASH

	//Found to be fast and have very good distribution
	private static long newCompatHashingAlg( String key ) {
		CRC32 checksum = new CRC32();
		checksum.update( key.getBytes() );
		long crc = checksum.getValue();
		return (crc >> 16) & 0x7fff;
	}

4:CONSISTENT_HASH

	private static long md5HashingAlg( String key ) {
		MessageDigest md5 = MD5.get();//MessageDigest.getInstance( "MD5" )
		md5.reset();
		md5.update( key.getBytes() );
		byte[] bKey = md5.digest();
		long res = ((long)(bKey[3]&0xFF) << 24) | ((long)(bKey[2]&0xFF) << 16) | ((long)(bKey[1]&0xFF) << 8) | (long)(bKey[0]&0xFF);
		return res;
	}

二：初始化一致性hash中的"圆圈"

private void populateConsistentBuckets() {
		//TreeMap来存储memcache服务器和它的hash值的对应关系；多个虚拟节点可以对应相同的服务器
		this.consistentBuckets = new TreeMap<Long,String>();
		//不同的memcache服务器可以配置不同的权值
		MessageDigest md5 = MD5.get();
		if ( this.totalWeight <= 0 && this.weights !=  null ) {
			for ( int i = 0; i < this.weights.length; i++ )
				this.totalWeight += ( this.weights[i] == null ) ? 1 : this.weights[i];
		}
		else if ( this.weights == null ) {
			this.totalWeight = this.servers.length;
		}
		for ( int i = 0; i < servers.length; i++ ) {
			int thisWeight = 1;
			if ( this.weights != null && this.weights[i] != null )
				thisWeight = this.weights[i];
			double factor = Math.floor( ((double)(40 * this.servers.length * thisWeight)) / (double)this.totalWeight );
			for ( long j = 0; j < factor; j++ ) {
				byte[] d = md5.digest( ( servers[i] + "-" + j ).getBytes() );
				for ( int h = 0 ; h < 4; h++ ) {
					Long k = 
						  ((long)(d[3+h*4]&0xFF) << 24)
						| ((long)(d[2+h*4]&0xFF) << 16)
						| ((long)(d[1+h*4]&0xFF) << 8)
						| ((long)(d[0+h*4]&0xFF));
					consistentBuckets.put( k, servers[i] );
					if ( log.isDebugEnabled() )
						log.debug( "++++ added " + servers[i] + " to server bucket" );
				}				
			}
		}
	}

三：存入和取出value时如何根据key确定memcache服务器

	//找到第一个大于或者等于hv的那个虚拟节点的hash值
	private Long findPointFor( Long hv ) {
		SortedMap<Long,String> tmap =
			this.consistentBuckets.tailMap( hv );
		return ( tmap.isEmpty() ) ? this.consistentBuckets.firstKey() : tmap.firstKey();
	}