有一堆待执行的任务(N个) 有一机器集群(M个),怎样分配任务最佳,使得每台机器分到的任务数尽量均衡,当机器集群数量发生变化时,任务数还是尽量平均分配,并且对于已分配的任务尽量减少再次分配,减少任务移动的成本。
直接哈希取余算法
选择任务的某一特征值,哈希取余来分配任务,第i个任务的特征为hash(i) 则其应该分配到机器编号为 hash(i)%M
优点:分配方式简单
缺点:特征值和哈希算法的选择非常重要,否则任务会分配不均衡,极端情况会分配到同一台机器;
当机器的数量发生变化时,增加或者减少,此时M值发生变化,为了保证负载均衡,需要重新进行计算,大部分任务都移动重新分配到其他机器,移动成本太高。
一致性哈希算法
现在来看看一致性哈希算法是如何解决该问题的,首先特征值和哈希算法的选择可以使用业界已有优秀的哈希算法 FNV1_32_HASH、KETAMA_HASH
任务N个 0<i<N 机器 M个 0<j<M
任务i的哈希值 hash(i) 机器j的哈希值 hash(j)
一致性哈希分配策略
将M个机器依次计算哈希值 防止于一个0-2^32-1的环上,使用优秀的哈希算法可以保证M个机器在环上的分配尽可能的均匀。
对于每一个任务I 计算hash(i) 将其置于环上,顺时针找到第一个机器节点即为该任务待分配的物理节点。
如下所示
假定机器编号为 t1 t2 t3 任务编号为 k1 k2 k3 k4
按照选择规则可知任务分配策略为
任务 | K1 | K2 | K3 | K4 |
机器 | T1 | T2 | K1 | T3 |
顺时针寻找 即寻找比hash(i)大的最小的hash(j)机器编号
如果找不到 则取机器0号(K3情况)
机器数量增加
如果增加机器T4 则任务K3需要重新分配 由原来的T1变为T4 其他任务不需要重新分配,即每增加一台机器,需要改动的任务约占N*(1/M)
机器数量减少
如果机器T3下线,则任务K4需要由原来的T3变为T2,即需要改动的任务约占N*(1/M)
虚拟节点
当机器数量增加时,T4增加 只是将T1的负载分担一部分,T2和 T3并没有因为 T4的加入而减少负载压力。
可以通过引入虚拟节点来解决负载不均衡的问题。即将每台物理服务器虚拟为一组虚拟服务器,将虚拟服务器放置到哈希环上,如果要确定对象的服务器,需先确定对象的虚拟服务器,再由虚拟服务器确定物理服务器。
对于每一台物理机器,增加了很多虚拟节点 这样当增加一台机器时,会增加很多虚拟节点,这些虚拟节点会尽可能分部在哈希环上,分担尽可能多的任务负载。
下面是机器数和虚拟节点数的一些最佳实践,横轴为机器数 纵轴为虚拟节点数
假定10台机器 100000个任务 研究一下各种算法下 机器数量增加减少时,任务分配情况如何变化
一致性哈希算法编码实践
代码如下:
/**
* 一致性哈希算法
*
* @author c00522789
* @since 2022/1/22
*/
public class ConsistentHashLoadBalanceNoVirtualNode {
private TreeMap<Long, String> realNodes = new TreeMap<>();
private String[] nodes;
public ConsistentHashLoadBalanceNoVirtualNode(String[] nodes) {
this.nodes = Arrays.copyOf(nodes, nodes.length);
init();
}
private void init() {
for (String nodeName: nodes) {
realNodes.put(hash(nodeName, 0), nodeName);
}
}
private Long hash(String nodeName, int number) {
byte[] digest = md5(nodeName);
return (((long) (digest[3 + number * 4] & 0xFF) << 24)
| ((long) (digest[2 + number * 4] & 0xFF) << 16)
| ((long) (digest[1 + number * 4] & 0xFF) << 8)
| (digest[number * 4] & 0xFF))
& 0xFFFFFFFFL;
}
public String selectNode(String key) {
Long hashKey = hash(key, 0);
if (realNodes.containsKey(hashKey)) {
return realNodes.get(hashKey);
}
// 比hashKey大的最小值
Map.Entry<Long, String> entry = realNodes.ceilingEntry(hashKey);
if (entry == null) {
return nodes[0];
}
return entry.getValue();
}
private void printTreeNode() {
if (realNodes == null || realNodes.isEmpty()) {
System.out.println("empty");
return;
}
realNodes.forEach((hashKey, node) -> {
System.out.println(new StringBuffer().append(node).append("==>").append(hashKey));
});
}
/**
* 负载分配算法
* 任务尽可能分配均衡
* 机器减少时 任务重新分配 尽量均衡 已分配的任务尽量少的重新分配
* 机器增加时 任务重新分配 尽量均衡 已分配的任务尽量少的重新分配
* */
public static void main(String[] args) {
decrease();
increase();
}
private static void decrease() {
String[] nodes = new String[10];
for (int n=0; n<10; n++) {
nodes[n] = "192.168.2." + n + ":8080";
}
for (int n=10; n>0; n--) {
ConsistentHashLoadBalanceNoVirtualNode consistentHash =
new ConsistentHashLoadBalanceNoVirtualNode(Arrays.copyOf(nodes, n));
Map<String, Integer> map = new HashMap<>();
for (int i=0; i<100000; i++) {
String key = consistentHash.selectNode(String.valueOf(i));
map.put(key, map.getOrDefault(key, 0) + 1);
}
System.out.println(map);
}
}
private static void increase() {
String[] nodes = new String[10];
for (int n=0; n<10; n++) {
nodes[n] = "192.168.2." + n + ":8080";
}
for (int n=1; n<=10; n++) {
ConsistentHashLoadBalanceNoVirtualNode consistentHash =
new ConsistentHashLoadBalanceNoVirtualNode(Arrays.copyOf(nodes, n));
// consistentHash.printTreeNode();
Map<String, Integer> map = new HashMap<>();
for (int i=0; i<100000; i++) {
String key = consistentHash.selectNode(String.valueOf(i));
map.put(key, map.getOrDefault(key, 0) + 1);
}
System.out.println(map);
}
}
}
运算结果如下:
机器数由10台逐渐减少
{192.168.2.5:8080=17548, 192.168.2.7:8080=17938, 192.168.2.1:8080=19770, 192.168.2.6:8080=8536, 192.168.2.2:8080=1805, 192.168.2.9:8080=2694, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.8:8080=3848, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.7:8080=17938, 192.168.2.1:8080=22464, 192.168.2.6:8080=8536, 192.168.2.2:8080=1805, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.8:8080=3848, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.7:8080=17938, 192.168.2.1:8080=26312, 192.168.2.6:8080=8536, 192.168.2.2:8080=1805, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.1:8080=26312, 192.168.2.6:8080=8536, 192.168.2.2:8080=19743, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.1:8080=26312, 192.168.2.2:8080=19743, 192.168.2.3:8080=9260, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.1:8080=26312, 192.168.2.2:8080=19743, 192.168.2.3:8080=26808, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.1:8080=35196, 192.168.2.2:8080=19743, 192.168.2.3:8080=26808, 192.168.2.0:8080=18253} {192.168.2.1:8080=62004, 192.168.2.2:8080=19743, 192.168.2.0:8080=18253} {192.168.2.1:8080=62004, 192.168.2.0:8080=37996} {192.168.2.0:8080=100000} |
10台机器逐渐增加
{192.168.2.0:8080=100000} {192.168.2.1:8080=62004, 192.168.2.0:8080=37996} {192.168.2.1:8080=62004, 192.168.2.2:8080=19743, 192.168.2.0:8080=18253} {192.168.2.1:8080=35196, 192.168.2.2:8080=19743, 192.168.2.3:8080=26808, 192.168.2.0:8080=18253} {192.168.2.1:8080=26312, 192.168.2.2:8080=19743, 192.168.2.3:8080=26808, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.1:8080=26312, 192.168.2.2:8080=19743, 192.168.2.3:8080=9260, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.1:8080=26312, 192.168.2.6:8080=8536, 192.168.2.2:8080=19743, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.7:8080=17938, 192.168.2.1:8080=26312, 192.168.2.6:8080=8536, 192.168.2.2:8080=1805, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.7:8080=17938, 192.168.2.1:8080=22464, 192.168.2.6:8080=8536, 192.168.2.2:8080=1805, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.8:8080=3848, 192.168.2.4:8080=8884} {192.168.2.5:8080=17548, 192.168.2.7:8080=17938, 192.168.2.1:8080=19770, 192.168.2.6:8080=8536, 192.168.2.2:8080=1805, 192.168.2.9:8080=2694, 192.168.2.3:8080=724, 192.168.2.0:8080=18253, 192.168.2.8:8080=3848, 192.168.2.4:8080=8884} |
虚拟节点一致性哈希算法编码实践
代码如下:
/**
* 一致性哈希算法--虚拟节点
*
* @author c00522789
* @since 2022/1/22
*/
public class ConsistentHashLoadBalance {
private TreeMap<Long, String> virtualNodes = new TreeMap<>();
private LinkedList<String> nodes;
// 每台物理机器的虚拟节点的数量
private final int replicCnt;
public ConsistentHashLoadBalance(LinkedList<String> nodes, int replicCnt) {
this.nodes = nodes;
this.replicCnt = replicCnt;
init();
}
private void init() {
for (String nodeName : nodes) {
for (int i = 0; i < replicCnt / 4; i++) {
String virtualNodeName = getNodeNameByIndex(nodeName, i);
for (int j = 0; j < 4; j++) {
virtualNodes.put(hash(virtualNodeName, j), nodeName);
}
}
}
}
private String getNodeNameByIndex(String nodeName, int index) {
return new StringBuilder(nodeName).append("&&").append(index).toString();
}
private Long hash(String nodeName, int number) {
byte[] digest = md5(nodeName);
return (((long) (digest[3 + number * 4] & 0xFF) << 24) | ((long) (digest[2 + number * 4] & 0xFF) << 16) | (
(long) (digest[1 + number * 4] & 0xFF) << 8) | (digest[number * 4] & 0xFF)) & 0xFFFFFFFFL;
}
public String selectNode(String key) {
Long hashKey = hash(key, 0);
if (virtualNodes.containsKey(hashKey)) {
return virtualNodes.get(hashKey);
}
Map.Entry<Long, String> entry = virtualNodes.ceilingEntry(hashKey);
if (entry == null) {
return nodes.getFirst();
}
return entry.getValue();
}
private void printTreeNode() {
if (virtualNodes == null || virtualNodes.isEmpty()) {
System.out.println("empty");
return;
}
virtualNodes.forEach((hashKey, node) -> {
System.out.println(new StringBuffer().append(node).append("==>").append(hashKey));
});
}
public static void main(String[] args) {
decrease();
increase();
}
private static void decrease() {
LinkedList<String> nodes = new LinkedList<>();
for (int n=0; n<10; n++) {
nodes.add("192.168.2." + n + ":8080");
}
for (int n=10; n>1; n--) {
nodes.removeLast();
ConsistentHashLoadBalance consistentHash =
new ConsistentHashLoadBalance(nodes, 10);
// consistentHash.printTreeNode();
Map<String, Integer> map = new HashMap<>();
for (int i=0; i<100000; i++) {
String key = consistentHash.selectNode(String.valueOf(i));
map.put(key, map.getOrDefault(key, 0) + 1);
}
System.out.println(map);
}
}
private static void increase() {
LinkedList<String> nodes = new LinkedList<>();
for (int n=0; n<10; n++) {
nodes.add("192.168.2." + n + ":8080");
ConsistentHashLoadBalance consistentHash =
new ConsistentHashLoadBalance(nodes, 20);
// consistentHash.printTreeNode();
Map<String, Integer> map = new HashMap<>();
for (int i=0; i<100000; i++) {
String key = consistentHash.selectNode(String.valueOf(i));
map.put(key, map.getOrDefault(key, 0) + 1);
}
System.out.println(map);
}
}
}
运算结果如下:
机器数由10台逐渐减少
{192.168.2.7:8080=12759, 192.168.2.5:8080=7687, 192.168.2.1:8080=8890, 192.168.2.6:8080=16898, 192.168.2.2:8080=8004, 192.168.2.3:8080=7663, 192.168.2.0:8080=8632, 192.168.2.4:8080=17784, 192.168.2.8:8080=11683} {192.168.2.7:8080=13416, 192.168.2.5:8080=9166, 192.168.2.1:8080=11424, 192.168.2.6:8080=16898, 192.168.2.2:8080=8004, 192.168.2.3:8080=13427, 192.168.2.0:8080=9317, 192.168.2.4:8080=18348} {192.168.2.5:8080=17508, 192.168.2.1:8080=15078, 192.168.2.6:8080=16898, 192.168.2.2:8080=8088, 192.168.2.3:8080=14274, 192.168.2.0:8080=9317, 192.168.2.4:8080=18837} {192.168.2.5:8080=20414, 192.168.2.1:8080=15111, 192.168.2.2:8080=8088, 192.168.2.3:8080=14440, 192.168.2.0:8080=18607, 192.168.2.4:8080=23340} {192.168.2.1:8080=17864, 192.168.2.2:8080=11225, 192.168.2.3:8080=14440, 192.168.2.0:8080=29163, 192.168.2.4:8080=27308} {192.168.2.1:8080=17864, 192.168.2.2:8080=11225, 192.168.2.3:8080=29875, 192.168.2.0:8080=41036} {192.168.2.1:8080=22396, 192.168.2.2:8080=32750, 192.168.2.0:8080=44854} {192.168.2.1:8080=48076, 192.168.2.0:8080=51924} {192.168.2.0:8080=100000} |
10台机器逐渐增加
{192.168.2.0:8080=100000} {192.168.2.1:8080=57930, 192.168.2.0:8080=42070} {192.168.2.1:8080=39211, 192.168.2.2:8080=24515, 192.168.2.0:8080=36274} {192.168.2.1:8080=32577, 192.168.2.2:8080=20091, 192.168.2.3:8080=18078, 192.168.2.0:8080=29254} {192.168.2.1:8080=19092, 192.168.2.2:8080=18875, 192.168.2.3:8080=16774, 192.168.2.0:8080=18893, 192.168.2.4:8080=26366} {192.168.2.5:8080=21527, 192.168.2.1:8080=16800, 192.168.2.2:8080=15388, 192.168.2.3:8080=15144, 192.168.2.0:8080=16051, 192.168.2.4:8080=15090} {192.168.2.5:8080=16308, 192.168.2.1:8080=14945, 192.168.2.6:8080=14046, 192.168.2.2:8080=15388, 192.168.2.3:8080=13763, 192.168.2.0:8080=13438, 192.168.2.4:8080=12112} {192.168.2.7:8080=10186, 192.168.2.5:8080=12335, 192.168.2.1:8080=14233, 192.168.2.6:8080=13941, 192.168.2.2:8080=15230, 192.168.2.3:8080=9747, 192.168.2.0:8080=13154, 192.168.2.4:8080=11174} {192.168.2.7:8080=6703, 192.168.2.5:8080=11749, 192.168.2.1:8080=13619, 192.168.2.6:8080=12437, 192.168.2.2:8080=14236, 192.168.2.3:8080=9304, 192.168.2.0:8080=12259, 192.168.2.8:8080=10835, 192.168.2.4:8080=8858} {192.168.2.7:8080=6172, 192.168.2.5:8080=10805, 192.168.2.1:8080=11395, 192.168.2.2:8080=13897, 192.168.2.6:8080=8876, 192.168.2.9:8080=10127, 192.168.2.3:8080=8671, 192.168.2.0:8080=11411, 192.168.2.8:8080=10321, 192.168.2.4:8080=8325} |
可知使用虚拟节点算法 任务的分配能够更加均衡
参考文档
https://segmentfault.com/a/1190000021199728