场景:
处理一个list集合:List<CharVar> dpsCharVars,过滤不同的映射集合,并进行下一步处理。
JSONObject onehot_mappings = new JSONObject();
JSONObject seq_mappings = new JSONObject();
JSONObject woe_mappings = new JSONObject();
List<CharVar> oneHotList = dpsCharVars.parallelStream()
.filter(CharVar::iisOneHotMapping)
.forEach(dpsChar -> onehot_mappings.put(dpsChar.getEnName(), JSONObject.parseObject(dpsChar.getVarCon(), List.class)));
log.info("====================onehot映射大小{} :{}==========================", onehot_mappings.size(), onehot_mappings);
//顺序替换的变量 的映射规则 {"col1":{"a":1}, "col2":{"b":2} }
List<CharVar> orderList = dpsCharVars.parallelStream()
.filter(CharVar::iisOrderMapping)
.forEach(dpsChar -> seq_mappings.put(dpsChar.getEnName(), JSONObject.parseObject(dpsChar.getVarCon())));
log.info("====================顺序映射大小{} :{}==========================", seq_mappings.size(), seq_mappings);
//按woe替换的变量 的映射规则 {"col3":{"a":1}, "col4":{"b":2} }
List<CharVar> woeList = dpsCharVars.parallelStream()
.filter(CharVar::iisWoeMapping)
.forEac(dpsChar -> woe_mappings.put(dpsChar.getEnName(), JSONObject.parseObject(dpsChar.getVarCon())));
log.info("====================woe映射大小{} :{}==========================", woe_mappings.size(), woe_mappings);
结果:
运行多次,不是onehot_mappings少一两个值,就是seq_mappings少一两个值。
原因:
这是因为使用了并发流parallelStream().
解决办法:
方法一:
将过滤出来的流先转成list,再进行计算
List<CharVar> oneHotList = dpsCharVars.parallelStream()
.filter(CharVar::iisOneHotMapping)
.collect(Collectors.toList());
oneHotList.forEach(dpsChar -> onehot_mappings.put(dpsChar.getEnName(), JSONObject.parseObject(dpsChar.getVarCon(), List.class)));
log.info("====================onehot映射大小{} :{}==========================", onehot_mappings.size(), onehot_mappings);
//顺序替换的变量 的映射规则 {"col1":{"a":1}, "col2":{"b":2} }
List<CharVar> orderList = dpsCharVars.parallelStream()
.filter(CharVar::iisOrderMapping)
.collect(Collectors.toList());
orderList.forEach(dpsChar -> seq_mappings.put(dpsChar.getEnName(), JSONObject.parseObject(dpsChar.getVarCon())));
log.info("====================顺序映射大小{} :{}==========================", seq_mappings.size(), seq_mappings);
//按woe替换的变量 的映射规则 {"col3":{"a":1}, "col4":{"b":2} }
List<CharVar> woeList = dpsCharVars.parallelStream()
.filter(CharVar::iisWoeMapping)
.collect(Collectors.toList());
woeList.forEach(dpsChar -> woe_mappings.put(dpsChar.getEnName(), JSONObject.parseObject(dpsChar.getVarCon())));
log.info("====================woe映射大小{} :{}==========================", woe_mappings.size(), woe_mappings);
方法二:推荐
dpsCharVars.forEach(var -> {
if (var.iisOneHotMapping()) {
onehot_mappings.put(var.getEnName(), JSONObject.parseObject(var.getVarCon(), List.class));
} else if (var.iisOrderMapping()) {
seq_mappings.put(var.getEnName(), JSONObject.parseObject(var.getVarCon()));
} else if (var.iisWoeMapping()) {
woe_mappings.put(var.getEnName(), JSONObject.parseObject(var.getVarCon()));
} else {
log.info("未知映射类型:{}", var);
}
});
log.info("====================onehot映射大小{} :{}==========================", onehot_mappings.size(), onehot_mappings);
log.info("====================顺序映射大小{} :{}==========================", seq_mappings.size(), seq_mappings);
log.info("====================woe映射大小{} :{}==========================", woe_mappings.size(), woe_mappings);