测试hash碰撞的概率
- 首先准备一个十万单词的txt文件,然后进行读取,存放到set集合中去重
- 然后遍历set集合将每一个单词求hashcode,然后存放到一个list中
- list总元素大小-List进行去重处理后的大小/去重前的大小=hash碰撞概率
/*求hashcode的函数*/
public Integer hashCode(Integer mutb,String str){
int hash = 0;
char[] chars = str.toCharArray();
for (int i = 0; i <chars.length ; i++) {
hash = hash*mutb+chars[i];
}
return hash;
}
/*计算hasn碰撞概率*/
public double probability(List<Integer> list){
//碰撞数
int crashnum = list.size()-(int) list.stream().distinct().count();
return (crashnum*1.0)/list.size();
}
结果:
单词数量:103976
乘数 = 2, 最小Hash = 97, 最大Hash = 1842581979, 碰撞数量 = 60382, 碰撞概率 = 58.0730%
乘数 = 3, 最小Hash = -2147308825, 最大Hash = 2146995420, 碰撞数量 = 24300, 碰撞概率 = 23.3708%
乘数 = 5, 最小Hash = -2147091606, 最大Hash = 2147227581, 碰撞数量 = 7994, 碰撞概率 = 7.6883%
乘数 = 7, 最小Hash = -2147431389, 最大Hash = 2147226363, 碰撞数量 = 3826, 碰撞概率 = 3.6797%
乘数 = 17, 最小Hash = -2147238638, 最大Hash = 2147101452, 碰撞数量 = 576, 碰撞概率 = 0.5540%
乘数 = 31, 最小Hash = -2147461248, 最大Hash = 2147444544, 碰撞数量 = 2, 碰撞概率 = 0.0019%
乘数 = 32, 最小Hash = -2007883634, 最大Hash = 2074238226, 碰撞数量 = 34947, 碰撞概率 = 33.6106%
乘数 = 33, 最小Hash = -2147469046, 最大Hash = 2147378587, 碰撞数量 = 1, 碰撞概率 = 0.0010%
乘数 = 39, 最小Hash = -2147463635, 最大Hash = 2147443239, 碰撞数量 = 0, 碰撞概率 = 0.0000%
乘数 = 41, 最小Hash = -2147423916, 最大Hash = 2147441721, 碰撞数量 = 1, 碰撞概率 = 0.0010%
乘数 = 199, 最小Hash = -2147459902, 最大Hash = 2147480320, 碰撞数量 = 0, 碰撞概率 = 0.0000%
Process finished with exit code 0
从结果可以看出
- 乘数是2时,hash的取值范围比较小,基本是堆积到一个范围内了。
- 乘数是3、5、7、17等,都有较大的碰撞概率
- 乘数是31的时候,碰撞的概率已经很小了,基本稳定。
- 乘数为199的碰撞概率更小但这个范围值已经远超过int的取值范围了,如果用此数作为乘数,又返回int值,就会丢失数据信息。
对hash值进行散列存放
- 将2的32次方分为64个存值区间
- 将hash值存放到这64个区间中
- 统计这64个区间,每个区间的hash值个数
public static Map<Integer, Integer> hashArea(Set<String> strList, Integer multiplier){
List<Integer> hashCodeList = new ArrayList<>();
for (String str : strList) {
Integer hashCode = hashCode(str, multiplier);
hashCodeList.add(hashCode);
}
return hashArea(hashCodeList);
}
public static Map<Integer, Integer> hashArea(List<Integer> hashCodeList) {
Map<Integer, Integer> statistics = new LinkedHashMap<>();
int start = 0;
//2的32次方分为64个区间
for (long i = 0x80000000; i <= 0x7fffffff; i += 67108864) {
long min = i;
long max = min + 67108864;
// 筛选出每个区间里的哈希值数量
int num = (int) hashCodeList.parallelStream().filter(x -> x >= min && x < max).count();
statistics.put(start++, num);
}
return statistics;
}
//测试
public void test_hashArea() {
System.out.println(HashCode.hashArea(words, 2).values());
System.out.println(HashCode.hashArea(words, 7).values());
System.out.println(HashCode.hashArea(words, 31).values());
System.out.println(HashCode.hashArea(words, 32).values());
System.out.println(HashCode.hashArea(words, 199).values());
}
结果
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 103910, 34, 0, 18, 0, 0, 7, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 2, 0, 0, 0, 0]
[310, 364, 284, 292, 393, 506, 656, 460, 451, 410, 441, 522, 485, 458, 725, 559, 391, 493, 319, 308, 479, 469, 454, 354, 491, 486, 361, 425, 347, 442, 245, 317, 38397, 15253, 286, 371, 869, 1509, 2012, 1049, 1186, 2890, 7992, 6821, 975, 1061, 1467, 1130, 1350, 1200, 482, 291, 319, 301, 241, 245, 324, 239, 260, 272, 433, 317, 266, 471]
[1292, 1065, 1197, 1148, 1217, 978, 1157, 1157, 1203, 1099, 1737, 3011, 2235, 2160, 1612, 2443, 2004, 1941, 3268, 2047, 1792, 1615, 1257, 1316, 1350, 1297, 1234, 1698, 1318, 1137, 1217, 1385, 8237, 8404, 1214, 1205, 1199, 1174, 1092, 1203, 1472, 1047, 1125, 1264, 1317, 1075, 1527, 1231, 1552, 1041, 1201, 1085, 1258, 1153, 1255, 1223, 1289, 1084, 1008, 1440, 1230, 1400, 1107, 1277]
[0, 0, 1607, 1044, 1614, 1559, 1135, 1575, 1664, 2360, 1733, 2642, 1305, 301, 195, 0, 0, 0, 3653, 1745, 3973, 2144, 2987, 1750, 1090, 3674, 2153, 2471, 1390, 391, 579, 0, 6948, 7109, 2489, 2931, 1340, 980, 1016, 2407, 5064, 2291, 3314, 4285, 1138, 351, 658, 0, 0, 0, 2142, 988, 2709, 485, 1919, 587, 547, 2290, 612, 1555, 870, 57, 160, 0]
[1415, 1455, 1617, 1642, 1491, 1418, 1506, 1663, 1694, 1594, 1450, 1428, 1462, 1545, 1693, 1566, 1354, 1413, 1495, 1720, 1648, 1641, 1473, 1386, 1401, 1468, 1548, 1518, 1342, 1409, 1561, 1656, 4110, 1423, 1319, 1327, 1404, 1639, 1579, 1389, 1406, 1452, 1613, 3019, 3373, 3030, 1748, 1471, 1490, 1469, 1457, 1396, 1457, 1644, 1754, 1624, 1503, 1498, 1304, 1507, 1458, 1501, 1486, 1454]
Process finished with exit code 0
从结果可以看出,除了199不能用以外,31的散列结果相对来说比较均匀。