在二手车业务线,现阶段无法实现车辆、人、车商信息的在业务审核流程中的数据查重应用,因此业务方为了达成这一目标,基于数据采集和数据查询,应运而生了关系图谱服务。
一、系统架构
从上述系统架构图我们可以看出:
- 1、关系图谱服务主要提供两种能力,数据采集和数据查重。
- 2、数据采集基于接入消息,消费业务线内部的消息通知,基于元数据进行分析并落库。
- 3、数据查重基于HTTP服务,对业务线提供场景的数据查重服务。
- 4、关系图谱服务内部引擎主要包括,数据采集、数据加工、数据切分、数据存储。
- 5、关系图谱服务一期目标基于当前业务量,基于MySQL数据库存储。
二、业务概述
从上述业务流程图我们可以看出:
- 1、数据采集,基于接入MQ消息,然后业务逻辑层分析数据,进行数据加工与切分,并存储到数据库,数据切分成业务表,主要包括(b_person_info和b_vehicle_info)。
- 2、数据查重,基于HTTP接口,外部根据关系图谱协议规范,定义查重参数,并返回命中数据。
三、方案设计
上图是根据业务梳理出不同场景阶段可以得到的数据,从上图我们可以得到如下结论
- 1、第一列为人信息的分类,包括:主贷人、配偶、担保人等。
- 2、第二列为数据维度,其中单元格不同颜色表示在不同场景阶段可以拿到数据。
- 3、基于MQ消息通知,我们从消息中可以获取到订单号,然后再查询三方数据获取相关数据项。
3.1、数据采集
数据采集主要完成对接MQ消息通知,以及历史数据的同步(上线时必须),然后对数据加工切分并落库。
上图是整个代码设计的UML类图,从上述图可以看出:
- 1、第一层的"应用场景模块"是两个类,一个是消息消费、一个是数据同步。
- 2、第二层的"事件监听模块",是按照业务场景划分,对于不同业务场景,然后监听器通知对应的处理类处理即可。
- 3、第三层的"数据处理模块",按照数据类型(即主贷人、配偶等)抽象出抽象类,以及派生的子类。
3.1.1、SourceInfoContext
该类主要封装数据处理器上下文数据信息。
/**
* @description: 来源信息上下文对象
* @Date : 2020/12/2 5:33 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class SourceInfoContext {
/**
* 订单号
*/
private String appCode;
/**
* 车商id
*/
private Long dealerId;
/**
* 业务场景
*/
private SceneEnum scene;
/**
* 车商提交信息
*/
private List<CarDealerInfoDTO> carDealers;
}
/**
* @description: 1001-秒批提交;1002-准入提交;1003-贷前提交;1004-车辆评估备注;2001-车商账号提交;
* @Date : 2020/11/27 4:08 PM
* @Author : 石冬冬-Seig Heil
*/
public enum SceneEnum implements EnumValue {
SECOND_BATCH_SUBMIT(1001,"秒批提交"),
ADMITTANCE_SUBMIT(1002,"准入提交"),
BEFORE_LOAN_SUBMIT(1003,"贷前提交"),
VEHICLE_EVALUATE_REMARK(1004,"车辆评估备注"),
MERCHANT_ACCOUNT_SUBMIT(2001,"车商账号提交"),
;
private int index;
private String value;
SceneEnum(int index, String value ){
this.value = value;
this.index = index;
}
@Override
public int getIndex() {
return index;
}
@Override
public String getName() {
return value;
}
/**
* 根据索引获取对象
* @param index
* @return
*/
public static SceneEnum getByIndex(int index){
return Stream.of(SceneEnum.values()).filter(each -> each.getIndex() == index).findFirst().get();
}
/**
* 根据索引获取名称
* @param index
* @return
*/
public static String getNameByIndex(int index){
SceneEnum find = Stream.of(SceneEnum.values()).filter(each -> each.getIndex() == index).findFirst().get();
return null == find ? "" : find.getName();
}
}
3.1.2、AbstractInfoHandler
该类主要抽象封装数据处理,是车辆、人、车商信息处理模块的上层抽象类。
/**
* @description: 来源信息数据处理接口
* @Date : 2020/12/2 6:34 PM
* @Author : 石冬冬-Seig Heil
*/
public interface SourceInfoHandle {
/**
* 数据组装和数据处理
* @param context
*/
void handle(SourceInfoContext context);
}
/**
* @description: 抽象信息处理器
* @Date : 2020/12/2 5:29 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractInfoHandler<T> implements SourceInfoHandle{
protected final static int NORMAL_DATA_STATUS_INDEX = DataStatusEnum.NORMAL.getIndex();
protected final static int DELETED_DATA_STATUS_INDEX = DataStatusEnum.DELETED.getIndex();
protected final static String EMPTY_VALUE = "-";
/**
* 上下文容器
*/
protected static final ThreadLocal<SourceInfoContext> CONTEXT = new ThreadLocal<>();
/**
* 来源类型
*/
protected final SourceTypeEnum sourceType;
/**
* 是否需要标记删除重复记录
*/
protected boolean removeDuplicate;
@Autowired
protected DiamondConfigProxy diamondConfigProxy;
@Autowired
protected SerialNoGenerator serialNoGenerator;
@Autowired
protected SourceInfoQuerier sourceInfoQuerier;
public AbstractInfoHandler(SourceTypeEnum sourceType,boolean removeDuplicate) {
this.sourceType = sourceType;
this.removeDuplicate = removeDuplicate;
}
/**
* 执行
* @param ctx
*/
public void execute(SourceInfoContext ctx){
try {
CONTEXT.set(ctx);
log.info("[execute]appCode={},Handler={},sourceType={}",ctx.getAppCode(),this.getClass().getSimpleName(),sourceType);
handle(ctx);
} catch (Exception e) {
log.error("[execute]appCode={},Handler={},sourceType={},ex",ctx.getAppCode(),this.getClass().getSimpleName(),sourceType,e);
}finally {
CONTEXT.remove();
}
}
/**
* 已存在的记录集合
* @param sourceRecord
* @param context
* @return
*/
protected abstract List<T> queryExistsList(T sourceRecord, SourceInfoContext context);
/**
* 把图谱记录写入库
* @param sourceRecord 三方查询数据
* @param context 上下文对象
*/
protected abstract void store(T sourceRecord,SourceInfoContext context);
/**
* 从配置获取对象属性字段
* @return
*/
protected List<String> getPropertiesFromConfig(){
return getPropertiesFromConfigWithScene(CONTEXT.get().getScene());
}
/**
* 从配置获取对象属性字段
* @return
*/
protected List<String> getPropertiesFromConfigWithScene(SceneEnum scene){
Map<SourceTypeEnum,Map<SceneEnum,List<String>>> mapping = JSON.parseObject(diamondConfigProxy.configGatherRules(),
new TypeReference<Map<SourceTypeEnum,Map<SceneEnum,List<String>>>>(){});
List<String> properties = mapping.get(sourceType).get(scene);
return properties;
}
/**
* 检查两个对象相关字段是否全部相同
* @param source 三方数据源对象
* @param target 关系图谱数据库对象
* @return
*/
protected boolean isSameWithAnyFields(T source,T target){
List<String> properties = getPropertiesFromConfig();
if(CollectionsTools.isEmpty(properties)){
return true;
}
Class sourceClazz = source.getClass();
Class targetClazz = target.getClass();
Field sourceFiled,targetFiled;
Object sourceFieldValue,targetFieldValue;
try {
for (String property : properties) {
sourceFiled = sourceClazz.getDeclaredField(property);
targetFiled = targetClazz.getDeclaredField(property);
sourceFiled.setAccessible(true);
targetFiled.setAccessible(true);
sourceFieldValue = sourceFiled.get(source);
targetFieldValue = targetFiled.get(target);
if(!Objects.equals(sourceFieldValue,targetFieldValue)) {
return false;
}
}
} catch (NoSuchFieldException e) {
log.error(e.getMessage(), e);
} catch (IllegalAccessException e) {
log.error(e.getMessage(), e);
}
return true;
}
/**
* 从目标对象拷贝到
* @param source 三方数据源对象
* @param target 关系图谱数据库对象
*/
protected void copy(T source,T target){
List<String> properties = getPropertiesFromConfig();
copy(source, target, properties);
}
protected void copy(Object source, Object target, List<String> properties){
if(CollectionsTools.isEmpty(properties)){
return;
}
Class sourceClazz = source.getClass();
Class targetClazz = target.getClass();
Field sourceFiled,targetFiled;
try {
for (String property : properties) {
sourceFiled = sourceClazz.getDeclaredField(property);
targetFiled = targetClazz.getDeclaredField(property);
sourceFiled.setAccessible(true);
targetFiled.setAccessible(true);
targetFiled.set(target,sourceFiled.get(source));
}
} catch (NoSuchFieldException e) {
log.error(e.getMessage(), e);
} catch (IllegalAccessException e) {
log.error(e.getMessage(), e);
}
}
/**
* 返回处理器类名称
* @return
*/
protected String getHandlerClassName(){
return this.getClass().getSimpleName();
}
/**
* 从上下文容器拿到业务场景
* @return
*/
protected SceneEnum getSceneFromContext(){
return null == CONTEXT.get() ? null : CONTEXT.get().getScene();
}
}
3.1.3、AbstractVehicleInfoHandler
该类主要抽象封装车辆信息的数据处理,只有一个派生类
VehicleInfoHandler
。
/**
* @description: 抽象自然人信息处理器
* @Date : 2020/12/3 11:32 AM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractVehicleInfoHandler extends AbstractInfoHandler<VehicleInfo> {
@Resource
protected VehicleInfoService vehicleInfoService;
public AbstractVehicleInfoHandler(SourceTypeEnum sourceType) {
super(sourceType,Boolean.FALSE);
}
@Override
protected void store(VehicleInfo record,SourceInfoContext context) {
if(Objects.nonNull(record)){
List<VehicleInfo> recordList = queryExistsList(record,context);
if(CollectionsTools.isNotEmpty(recordList)){
for (VehicleInfo sourceRecord : recordList) {
if(isSameWithAnyFields(sourceRecord.with(),record.with())){
return;
}
}
}
record.setDataStatus(NORMAL_DATA_STATUS_INDEX);
record.setDataCode(serialNoGenerator.generalVehicleInfoDataCode());
record.setScene(context.getScene().getIndex());
record.setSourceType(sourceType.getIndex());
log.info("[store][insertRecord]appCode={},Handler={},sourceType={}",record.getAppCode(),getHandlerClassName(),sourceType);
vehicleInfoService.insertRecord(record.with());
}
}
@Override
protected List<VehicleInfo> queryExistsList(VehicleInfo sourceRecord,SourceInfoContext context) {
VehicleInfoForm queryForm = VehicleInfoForm.builder()
.appCode(context.getAppCode())
.sourceType(sourceType.getIndex())
//.scene(context.getScene().getIndex())
.dataStatus(NORMAL_DATA_STATUS_INDEX)
.build();
return vehicleInfoService.queryList(queryForm);
}
}
/**
* @description: 车辆信息处理器
* @Date : 2020/12/2 6:22 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
@Slf4j
public class VehicleInfoHandler extends AbstractVehicleInfoHandler{
public VehicleInfoHandler() {
super(SourceTypeEnum.VEHICLE_INFO);
}
@Override
public void handle(SourceInfoContext context) {
VehicleInfo sourceRecord = sourceInfoQuerier.queryVehicleInfo(context.getAppCode());
SceneEnum scene = context.getScene();
if(SceneEnum.ADMITTANCE_SUBMIT == scene){
super.store(sourceRecord,context);
}
if(SceneEnum.BEFORE_LOAN_SUBMIT == scene || SceneEnum.VEHICLE_EVALUATE_REMARK == scene){
List<VehicleInfo> recordList = queryExistsList(sourceRecord,context);
Map<String,CarEvaluationBo> vinMapping = sourceInfoQuerier.queryVinMapping(context.getAppCode());
log.info("[EvaluateRemark],appCode={},vinMapping={}",context.getAppCode(), JSONObject.toJSONString(vinMapping));
recordList.forEach(each -> {
CarEvaluationBo apply = vinMapping.get(each.getVin());
if(Objects.nonNull(apply)){
log.info("[EvaluateRemark][Update],appCode={},vin={},remark={}",context.getAppCode(),each.getVin(),apply.getEvaluationRemarks());
each.setEvaluateRemark(StringTools.isNotEmpty(apply.getEvaluationRemarks()) ? apply.getEvaluationRemarks() : EMPTY_VALUE);
each.setModifiedTime(TimeTools.createNowTime());
vehicleInfoService.updateByPrimaryKeySelective(each);
}
});
}
}
}
3.1.4、AbstractCarDealerInfoHandler
该类主要抽象封装车商信息的数据处理,有三个派生类
CarDealerLegalPersonInfoHandler
、CarDealerPayeeInfoHandler
、CarDealerPrincipalInfoHandler
。
/**
* @description: 抽象车商信息处理器
* @Date : 2020/12/7 11:19 AM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractCarDealerInfoHandler extends AbstractInfoHandler<DealerInfo> {
@Resource
DealerInfoService dealerInfoService;
public AbstractCarDealerInfoHandler(SourceTypeEnum sourceType) {
super(sourceType, Boolean.TRUE);
}
/**
* 查询车商信息
* @param dealerId
* @return
*/
protected abstract List<DealerInfo> queryList(Long dealerId);
/**
* 是否应该处理
* @param record
* @return
*/
boolean shouldStore(DealerInfo record){
return Objects.nonNull(record) && StringTools.isNotEmpty(record.getName()) && !Objects.equals(record.getName(),EMPTY_VALUE);
}
@Override
protected void store(DealerInfo record,SourceInfoContext context) {
if(shouldStore(record)){
//是否删除历史雷同记录
if(removeDuplicate){
List<DealerInfo> recordList = queryExistsList(record,context);
if(CollectionsTools.isNotEmpty(recordList)){
for (DealerInfo sourceRecord : recordList) {
if(isSameWithAnyFields(sourceRecord.with(),record.with())){
return;
}
}
}
}
//存入新纪录
record.setDataCode(serialNoGenerator.generalDealerInfoDataCode());
record.setDataStatus(NORMAL_DATA_STATUS_INDEX);
record.setScene(context.getScene().getIndex());
record.setSourceType(sourceType.getIndex());
dealerInfoService.insertRecord(record.with());
}else{
log.debug("[execute]store NullObject,Handler={},sourceType={},context={}",
this.getClass().getSimpleName(),sourceType, JSONObject.toJSONString(context));
}
}
@Override
public void handle(SourceInfoContext context) {
List<DealerInfo> listWithoutSameFields = new ArrayList<>();
List<DealerInfo> list;
//倘若是车商账户提交推送的消息,则直接获取无需查询;否则,根据车商ID查询信息。
if(CollectionsTools.isEmpty(context.getCarDealers())){
list = queryList(context.getDealerId());
} else {
list = context.getCarDealers().stream().filter(each -> each.getSourceType().intValue() == sourceType.getIndex())
.map(
source -> DealerInfo.builder()
.externalId(Objects.toString(source.getExternalId()))
.appCode(Objects.toString(source.getMerchantId()))
.sourceType(source.getSourceType())
.name(source.getName())
.idNo(source.getIdNo())
.primaryMobile(source.getCellphone())
.companyAddressDetail(source.getDealerAddress())
.companyName(source.getDealerName())
.creditCardNo(source.getCreditCardNo())
.build()
).collect(Collectors.toList());
}
//本次添加的车商,去除某些字段值雷同的项
for (DealerInfo person : list) {
if(listWithoutSameFields.stream().filter(r -> isSameWithAnyFields(r, person)).count() > 0){
continue;
}
listWithoutSameFields.add(person);
}
log.info("[storeWithBatch]dealerId={},Handler={},sourceType={},list={}",context.getDealerId(),getHandlerClassName(),sourceType, JSONObject.toJSONString(list));
this.storeWithBatch(listWithoutSameFields,context);
}
@Override
protected List<DealerInfo> queryExistsList(DealerInfo sourceRecord, SourceInfoContext context) {
DealerInfoForm queryForm = DealerInfoForm.builder()
.sourceType(sourceType.getIndex())
.scene(context.getScene().getIndex())
.appCode(sourceRecord.getAppCode())
.dataStatus(NORMAL_DATA_STATUS_INDEX)
.build();
List<DealerInfo> recordList = dealerInfoService.queryList(queryForm);
return recordList;
}
/**
* 批量插入
* @param recordList
*/
protected void storeWithBatch(List<DealerInfo> recordList,SourceInfoContext context){
if(CollectionsTools.isEmpty(recordList)){
return;
}
for (DealerInfo record : recordList) {
store(record, context);
}
}
}
/**
* @description: 车商法人 (姓名、身份证号、手机号)
* @Date : 2020/12/2 5:26 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class CarDealerLegalPersonInfoHandler extends AbstractCarDealerInfoHandler {
public CarDealerLegalPersonInfoHandler() {
super(SourceTypeEnum.CAR_DEALER_LEGAL_PERSON);
}
@Override
protected List<DealerInfo> queryList(Long dealerId) {
return sourceInfoQuerier.queryCarDealerLegalInfo(dealerId);
}
}
/**
* @description: 车商收款人 (姓名、身份证号、手机号、银行卡号)
* @Date : 2020/12/2 5:26 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class CarDealerPayeeInfoHandler extends AbstractCarDealerInfoHandler {
public CarDealerPayeeInfoHandler() {
super(SourceTypeEnum.CAR_DEALER_PAYEE);
}
@Override
protected List<DealerInfo> queryList(Long dealerId) {
return sourceInfoQuerier.queryCarDealerPayeeInfo(dealerId);
}
}
/**
* @description: 车商负责人信息 (姓名、身份证号、手机号、单位名称、单位地址)
* @Date : 2020/12/2 5:26 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class CarDealerPrincipalInfoHandler extends AbstractCarDealerInfoHandler {
public CarDealerPrincipalInfoHandler() {
super(SourceTypeEnum.CAR_DEALER_PRINCIPAL);
}
@Override
protected List<DealerInfo> queryList(Long dealerId) {
return sourceInfoQuerier.queryCarDealerPrincipalInfo(dealerId);
}
}
3.1.5、AbstractPersonInfoHandler
该类主要抽象封装人信息的数据处理,有三个派生类
CreditorInfoHandler
、MateInfoHandler
等。
/**
* @description: 抽象自然人信息处理器
* @Date : 2020/12/3 11:32 AM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractPersonInfoHandler extends AbstractInfoHandler<PersonInfo>{
@Autowired
protected PersonInfoService personInfoService;
public AbstractPersonInfoHandler(SourceTypeEnum sourceType, boolean removeDuplicate) {
super(sourceType, removeDuplicate);
}
/**
* 是否应该处理
* @param record
* @return
*/
protected boolean shouldStore(PersonInfo record){
return Objects.nonNull(record) && StringTools.isNotEmpty(record.getName()) && !Objects.equals(record.getName(),EMPTY_VALUE);
}
@Override
protected void store(PersonInfo record,SourceInfoContext context) {
if(shouldStore(record)){
//是否删除历史雷同记录
if(removeDuplicate){
List<PersonInfo> recordList = queryExistsList(record,context);
if(CollectionsTools.isNotEmpty(recordList)){
for (PersonInfo sourceRecord : recordList) {
if(isSameWithAnyFields(sourceRecord.with(),record.with())){
return;
}
}
}
}
//存入新纪录
record.setDataCode(serialNoGenerator.generalPersonInfoDataCode());
record.setDataStatus(NORMAL_DATA_STATUS_INDEX);
record.setScene(context.getScene().getIndex());
record.setSourceType(sourceType.getIndex());
personInfoService.insertRecord(record.with());
}else{
log.debug("[execute]store NullObject,Handler={},sourceType={},context={}",
this.getClass().getSimpleName(),sourceType, JSONObject.toJSONString(context));
}
}
@Override
protected List<PersonInfo> queryExistsList(PersonInfo sourceRecord, SourceInfoContext context) {
PersonInfoForm queryForm = PersonInfoForm.builder()
.sourceType(sourceType.getIndex())
.scene(context.getScene().getIndex())
.appCode(context.getAppCode())
.dataStatus(NORMAL_DATA_STATUS_INDEX)
.build();
List<PersonInfo> recordList = personInfoService.queryList(queryForm);
return recordList;
}
/**
* 批量插入
* @param recordList
*/
protected void storeWithBatch(List<PersonInfo> recordList,SourceInfoContext context){
if(CollectionsTools.isEmpty(recordList)){
return;
}
recordList.forEach(each -> {
log.info("[storeWithBatch]appCode={},Handler={},sourceType={}",each.getAppCode(),getHandlerClassName(),sourceType);
each.setDataCode(serialNoGenerator.generalPersonInfoDataCode());
each.setDataStatus(NORMAL_DATA_STATUS_INDEX);
each.setSourceType(sourceType.getIndex());
each.setScene(context.getScene().getIndex());
each.with();
});
personInfoService.batchInsert(recordList);
}
}
/**
* @description: 主贷人信息 (姓名、身份证号、手机号、单位名称、单位地址、座机号、户籍地址、居住地址、居住电话、银行卡号)
* @Date : 2020/12/2 5:26 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class CreditorInfoHandler extends AbstractPersonInfoHandler {
public CreditorInfoHandler() {
super(SourceTypeEnum.CREDITOR_INFO, Boolean.FALSE);
}
@Override
public void handle(SourceInfoContext context) {
//查询新的修改记录
PersonInfo record = sourceInfoQuerier.queryCreditorInfo(context.getAppCode());
if(!shouldStore(record)){
return;
}
record.with();
//删除【当前场景】【之前场景】的【子集数据】
List<String> propertiesSecond = super.getPropertiesFromConfigWithScene(SceneEnum.SECOND_BATCH_SUBMIT);
PersonInfoForm propertiesSecondForm = PersonInfoForm.builder().appCode(context.getAppCode())
.sourceType(sourceType.getIndex())
.scene(SceneEnum.SECOND_BATCH_SUBMIT.getIndex())
.dataStatus(NORMAL_DATA_STATUS_INDEX)
.build();
List<String> propertiesAdmittance = super.getPropertiesFromConfigWithScene(SceneEnum.ADMITTANCE_SUBMIT);
PersonInfoForm propertiesAdmittanceForm = PersonInfoForm.builder().appCode(context.getAppCode())
.sourceType(sourceType.getIndex())
.scene(SceneEnum.ADMITTANCE_SUBMIT.getIndex())
.dataStatus(NORMAL_DATA_STATUS_INDEX)
.build();
List<String> propertiesBeforeLoan = super.getPropertiesFromConfigWithScene(SceneEnum.BEFORE_LOAN_SUBMIT);
PersonInfoForm propertiesBeforeLoanForm = PersonInfoForm.builder().appCode(context.getAppCode())
.sourceType(sourceType.getIndex())
.scene(SceneEnum.BEFORE_LOAN_SUBMIT.getIndex())
.dataStatus(NORMAL_DATA_STATUS_INDEX)
.build();
switch (context.getScene()){
case SECOND_BATCH_SUBMIT:
//软删除【秒批提交】子集
copy(record, propertiesSecondForm, propertiesSecond);
personInfoService.updateByQuery(propertiesSecondForm);
break;
case ADMITTANCE_SUBMIT:
//软删除【秒批提交】子集
copy(record, propertiesSecondForm, propertiesSecond);
personInfoService.updateByQuery(propertiesSecondForm);
//软删除【准入提交】子集
propertiesAdmittance.addAll(propertiesSecond);
copy(record, propertiesAdmittanceForm, propertiesAdmittance);
personInfoService.updateByQuery(propertiesAdmittanceForm);
break;
case BEFORE_LOAN_SUBMIT:
//软删除【秒批提交】子集
copy(record, propertiesSecondForm, propertiesSecond);
personInfoService.updateByQuery(propertiesSecondForm);
//软删除【准入提交】子集
propertiesAdmittance.addAll(propertiesSecond);
copy(record, propertiesAdmittanceForm, propertiesAdmittance);
personInfoService.updateByQuery(propertiesAdmittanceForm);
//软删除【贷前提交】子集
propertiesBeforeLoan.addAll(propertiesAdmittance);
copy(record, propertiesBeforeLoanForm, propertiesBeforeLoan);
personInfoService.updateByQuery(propertiesBeforeLoanForm);
break;
default:
break;
}
//补录新的修改记录
super.store(record,context);
}
}
/**
* @description: 配偶信息 (姓名、身份证号、手机号、单位名称)
* @Date : 2020/12/2 5:26 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class MateInfoHandler extends AbstractPersonInfoHandler {
public MateInfoHandler() {
super(SourceTypeEnum.MATE, Boolean.TRUE);
}
@Override
public void handle(SourceInfoContext context) {
PersonInfo record = sourceInfoQuerier.queryMateInfo(context.getAppCode());
super.store(record,context);
}
}
/**
* @description: 担保人信息 (姓名、身份证号、手机号、单位名称、单位地址、座机号、户籍地址)
* @Date : 2020/12/2 5:26 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class GuarantorInfoHandler extends AbstractPersonInfoHandler {
public GuarantorInfoHandler() {
super(SourceTypeEnum.GUARANTOR_INFO, Boolean.TRUE);
}
@Override
public void handle(SourceInfoContext context) {
PersonInfo record = sourceInfoQuerier.queryGuarantorInfo(context.getAppCode());
super.store(record,context);
}
}
3.1.6、SourceInfoQuerier
该类主要抽象对service查询数据的获取,并封装处理器内部所需的POJO。
/**
* @description: 来源信息查询器
* @Date : 2020/12/3 10:55 AM
* @Author : 石冬冬-Seig Heil
*/
@Component
@Slf4j
public class SourceInfoQuerier {
//省略依赖注入Service
/**
* 查询主贷人信息
* @param appCode
* @return
*/
public PersonInfo queryCreditorInfo(String appCode){
CustomerPersonInfo record = customerPersonInfoService.queryByAppCode(appCode);
if(Objects.isNull(record)){
return null;
}
List<CustomerCardInfo> customerCardInfos = customerCardInfoService.queryList(CustomerCardInfoForm.builder().appCode(appCode).build());
CustomerCardInfo customerCardInfo = CollectionsTools.isNotEmpty(customerCardInfos) ? customerCardInfos.get(0) : customerCardInfo_NULL;
return PersonInfo.builder()
.externalId(Objects.toString(record.getId()))
.appCode(appCode)
.name(record.getName())
.idNo(record.getIdNumber())
.primaryMobile(record.getMobile())
.secondMobile(record.getMobile2())
.companyName(record.getNowCompany())
.companyAddressOrigin(record.getNowUnitAddress())
.companyAddressProvince(record.getNowUnitProvinceName())
.companyAddressCity(record.getNowUnitCityName())
.companyAddressDistrict(record.getNowUnitDistrictName())
.companyAddressDetail(record.getNowUnitAddress())
.companyTelephone(record.getNowUnitTel())
.censusAddressOrigin(record.getHometownAddress())
.censusAddressProvince(record.getHometownProvinceName())
.censusAddressCity(record.getHometownCityName())
.censusAddressDistrict(record.getHometownDistrictName())
.censusAddressDetail(record.getHometownAddress())
.residenceAddressOrigin(record.getResidenceAddress())
.residenceAddressProvince(record.getResidenceProvinceName())
.residenceAddressCity(record.getResidenceCityName())
.residenceAddressDistrict(record.getResidenceDistrictName())
.residenceAddressDetail(record.getResidenceAddress())
.residenceTelephone(record.getResidenceTel())
.creditCardNo(Objects.nonNull(customerCardInfo) ? customerCardInfo.getRepAccountNo() : EMPTY_VALUE)
.build();
}
/**
* 查询担保人信息
* @param appCode
* @return
*/
public PersonInfo queryGuarantorInfo(String appCode){
CustomerRelatedInfo record = customerRelatedInfoService.queryByAppCode(appCode);
if(Objects.isNull(record)){
return null;
}
return PersonInfo.builder()
.externalId(Objects.toString(record.getId()))
.appCode(appCode)
.name(record.getDbName())
.idNo(record.getDbIdNo())
.primaryMobile(record.getDbMobile())
.companyName(record.getDbNowCompany())
.companyAddressOrigin(record.getDbNowUnitAddress())
.companyAddressProvince(record.getDbNowUnitProvinceName())
.companyAddressCity(record.getDbNowUnitCityName())
.companyAddressDistrict(record.getDbNowUnitDistrictName())
.companyAddressDetail(record.getDbNowUnitAddress())
//.companyTelphone()
.censusAddressOrigin(record.getDbAddress())
.censusAddressProvince(record.getDbProvinceName())
.censusAddressCity(record.getDbCityName())
.censusAddressDistrict(record.getDbDistrictName())
.censusAddressDetail(record.getDbAddress())
.build();
}
//省略其他成员方法
}
3.1.7、SubmitEventContext
/**
* @description: 提交事件上下文对象
* @Date : 2020/12/2 6:29 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class SubmitEventContext {
/**
* 订单号
*/
@NotNull
private String appCode;
/**
* 车商ID
*/
@NotNull
private Long merchantId;
/**
* 业务场景
*/
@NotNull
private SceneEnum scene;
/**
* 车商提交信息
*/
private List<CarDealerInfoDTO> carDealers;
}
3.1.8、AbstractSubmitListener
监听器抽象类,维护监听器通知的数据处理器成员。
/**
* @description: 提交事件监听接口
* @Date : 2020/12/2 6:29 PM
* @Author : 石冬冬-Seig Heil
*/
public interface SubmitEventListen {
/**
* 订阅事件
* @param context
*/
void subscribe(SubmitEventContext context);
}
/**
* @description: 抽象提交 事件监听器
* @Date : 2020/12/2 5:19 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractSubmitListener implements SubmitEventListen {
/**
* 关注场景
*/
protected SceneEnum sceneEnum;
/**
* 处理器集合
*/
protected List<AbstractInfoHandler> handlers = Lists.newArrayListWithExpectedSize(20);
public AbstractSubmitListener(SceneEnum sceneEnum) {
this.sceneEnum = sceneEnum;
}
@Override
public void subscribe(SubmitEventContext context) {
String appCode = context.getAppCode();
if(context.getScene() == sceneEnum){
log.info("[SubmitEventListen],appCode={},handlers={}",appCode,handlers.toArray());
handlers.forEach(each -> {
SourceInfoContext sourceInfoContext = SourceInfoContext.builder()
.appCode(appCode)
.dealerId(context.getMerchantId())
.carDealers(context.getCarDealers())
.scene(context.getScene())
.build();
each.execute(sourceInfoContext);
});
}
}
}
/**
* @description: 准入提交 事件监听器
* @Date : 2020/12/2 5:17 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class AdmittanceSubmitListener extends AbstractSubmitListener {
@Resource
CreditorInfoHandler creditorInfoHandler;
@Resource
MateInfoHandler mateInfoHandler;
@Resource
GuarantorInfoHandler guarantorInfoHandler;
@Resource
SellerInfoOfOldCarHandler sellerInfoOfOldCarHandler;
@Resource
EmergencyContactOneInfoHandler emergencyContactOneInfoHandler;
@Resource
EmergencyContactTwoInfoHandler emergencyContactTwoInfoHandler;
@Resource
VehicleInfoHandler vehicleInfoHandler;
public AdmittanceSubmitListener() {
super(SceneEnum.ADMITTANCE_SUBMIT);
}
@PostConstruct
void init(){
this.handlers.add(creditorInfoHandler);
this.handlers.add(mateInfoHandler);
this.handlers.add(guarantorInfoHandler);
this.handlers.add(sellerInfoOfOldCarHandler);
this.handlers.add(emergencyContactOneInfoHandler);
this.handlers.add(emergencyContactTwoInfoHandler);
this.handlers.add(vehicleInfoHandler);
}
}
/**
* @description: 贷前提交 事件监听器
* @Date : 2020/12/2 5:17 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class BeforeLoanSubmitListener extends AbstractSubmitListener {
@Resource
CreditorInfoHandler creditorInfoHandler;
@Resource
VehicleInfoHandler vehicleInfoHandler;
public BeforeLoanSubmitListener() {
super(SceneEnum.BEFORE_LOAN_SUBMIT);
}
@PostConstruct
void init(){
this.handlers.add(creditorInfoHandler);
this.handlers.add(vehicleInfoHandler);
}
}
/**
* @description: 车商账号提交 事件监听器
* @Date : 2020/12/2 5:17 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class MerchantSubmitListener extends AbstractSubmitListener {
@Resource
CarDealerLegalPersonInfoHandler carDealerLegalPersonInfoHandler;
@Resource
CarDealerPrincipalInfoHandler carDealerPrincipalInfoHandler;
@Resource
CarDealerPayeeInfoHandler carDealerPayeeInfoHandler;
public MerchantSubmitListener() {
super(SceneEnum.MERCHANT_ACCOUNT_SUBMIT);
}
@PostConstruct
void init(){
this.handlers.add(carDealerLegalPersonInfoHandler);
this.handlers.add(carDealerPrincipalInfoHandler);
this.handlers.add(carDealerPayeeInfoHandler);
}
}
/**
* @description: 秒批提交 事件监听器
* @Date : 2020/12/2 5:17 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class SecondBatchSubmitListener extends AbstractSubmitListener {
@Resource
CreditorInfoHandler creditorInfoHandler;
@Resource
SellerInfoHandler sellerInfoHandler;
@Resource
SalesManagerInfoHandler salesManagerInfoHandler;
public SecondBatchSubmitListener() {
super(SceneEnum.SECOND_BATCH_SUBMIT);
}
@PostConstruct
void init(){
this.handlers.add(creditorInfoHandler);
this.handlers.add(sellerInfoHandler);
this.handlers.add(salesManagerInfoHandler);
}
}
/**
* @description: 车辆评估备注 事件监听器
* 关注订单状态:
* 1、复审退回至授信-2101、
* 2、二次风控(终审)拒绝-3910、
* 3、等待贷前审核(已提交贷前资料)-4000
* @Date : 2020/12/29 3:22 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class VehicleEvaluateRemarkListener extends AbstractSubmitListener {
@Resource
VehicleInfoHandler vehicleInfoHandler;
public VehicleEvaluateRemarkListener() {
super(SceneEnum.VEHICLE_EVALUATE_REMARK);
}
@PostConstruct
void init(){
this.handlers.add(vehicleInfoHandler);
}
}
3.1.9、SubmitEventMulticaster
该类基于观察者模式,通知所有监听器处理,诸如
SecondBatchSubmitListener
、AdmittanceSubmitListener
等。
/**
* @description: 提交事件广播器
* @Date : 2020/12/2 6:45 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
@Slf4j
public class SubmitEventMulticaster {
/**
* 监听器成员
*/
protected final List<SubmitEventListen> listeners = new ArrayList<>();
@Resource
SecondBatchSubmitListener secondBatchSubmitListener;
@Resource
AdmittanceSubmitListener admittanceSubmitListener;
@Resource
BeforeLoanSubmitListener beforeLoanSubmitListener;
@Resource
MerchantSubmitListener merchantAccountSubmitEvent;
@Resource
VehicleEvaluateRemarkListener vehicleEvaluateRemarkListener;
@PostConstruct
void init(){
listeners.add(secondBatchSubmitListener);
listeners.add(admittanceSubmitListener);
listeners.add(beforeLoanSubmitListener);
listeners.add(merchantAccountSubmitEvent);
listeners.add(vehicleEvaluateRemarkListener);
}
/**
* 对外暴露接口
* @param context 上下文
*/
public void execute(SubmitEventContext context){
log.info("[execute]appCode={},ctx={}",context.getAppCode(),JSONObject.toJSONString(context));
listeners.forEach(event -> event.subscribe(context));
}
}
3.1.10、HistoryDataSynchronizer
历史数据同步器
/**
* @description: 历史数据(订单|车商)同步器
* @Date : 2020/12/7 11:49 AM
* @Author : 石冬冬-Seig Heil
*/
@Component
@Slf4j
public class HistoryDataSynchronizer {
@Resource
SubmitEventMulticaster submitEventMulticaster;
@Resource
OrderInfoService orderInfoService;
@Resource
DealerService dealerService;
@Resource
ExecutorService synchronizeThreadPoolExecutor;
/**
* 同步订单
* @param request
*/
public void synchronizeOrders(Request<String> request){
boolean byScope = CollectionsTools.isNotEmpty(request.getScope());
List<OrderInfo.SimpleEntity> recordList = byScope ? orderInfoService.queryTargets(request.getScope()) : orderInfoService.queryAll();
Map<String,Integer> orderMap = recordList.stream()
.collect(Collectors.toMap(OrderInfo.SimpleEntity::getAppCode,OrderInfo.SimpleEntity::getStatus));
Set<String> appCodes = orderMap.keySet();
if(request.getLimit() > 0){
appCodes = appCodes.stream().limit(request.getLimit()).collect(Collectors.toSet());
}
//订单3种场景
List<SceneEnum> sceneEnumList;
List<SubmitEventContext> contextList;
SubmitEventContext context;
StopWatch stopWatch = new StopWatch();
stopWatch.start("synchronizeOrders");
log.info("[synchronizeOrders]count={}",appCodes.size());
LongAdder adder = new LongAdder();
//遍历订单执行 3种场景场景事件
for(String appCode : appCodes){
Integer status = orderMap.get(appCode);
sceneEnumList = OrderScene.getScene(status);
if(!CollectionUtils.isEmpty(sceneEnumList)){
contextList = new ArrayList<>(sceneEnumList.size());
for (SceneEnum sceneEnum: sceneEnumList) {
context = SubmitEventContext.builder()
.appCode(appCode)
.scene(sceneEnum)
.build();
contextList.add(context);
adder.increment();
log.info("[synchronizeOrders]current={},appCode={},context={}",adder.intValue(), appCode, JSON.toJSONString(context));
//asyncCall(context);
}
asyncCall(contextList);
}else {
log.warn("[synchronizeOrders][sceneEnum] warning!!! appCode={},status={}", appCode, status);
}
}
stopWatch.stop();
log.info("[synchronizeOrders]finished,total={},duration={}",adder.intValue(),stopWatch.getLastTaskTimeMillis() / 1000);
}
/**
* 同步车商
* @param request
*/
public void synchronizeDealers(Request<Long> request){
boolean byScope = CollectionsTools.isNotEmpty(request.getScope());
List<Long> ids = byScope ? dealerService.queryTargets(request.getScope()) : dealerService.queryAllIds();
if(request.getLimit() > 0){
ids = ids.stream().limit(request.getLimit()).collect(Collectors.toList());
}
SubmitEventContext context;
StopWatch stopWatch = new StopWatch();
stopWatch.start();
log.info("[synchronizeDealers]count={}",ids.size());
LongAdder adder = new LongAdder();
for(Long merchantId : ids){
context = SubmitEventContext.builder()
.appCode(Objects.toString(merchantId))
.merchantId(merchantId)
.scene(SceneEnum.MERCHANT_ACCOUNT_SUBMIT)
.build();
adder.increment();
log.info("[synchronizeDealers]current={},merchantId={},context={}",adder.intValue(),merchantId, JSON.toJSONString(context));
asyncCall(context);
}
stopWatch.stop();
log.info("[synchronizeDealers]finished,total={},,duration={}",adder.intValue(),stopWatch.getLastTaskTimeMillis() / 1000);
}
/**
* 异步写入
* @param context
*/
void asyncCall(SubmitEventContext context){
try {
synchronizeThreadPoolExecutor.execute(() -> submitEventMulticaster.execute(context));
} catch (Exception e) {
log.error("[asyncCall]appCode={},context={}",context.getAppCode(),JSON.toJSONString(context));
}
}
void asyncCall(List<SubmitEventContext> contextList){
try {
synchronizeThreadPoolExecutor.execute(() -> {
for (SubmitEventContext context : contextList) {
submitEventMulticaster.execute(context);
}
});
} catch (Exception e) {
log.error("[asyncCall]size={},contextList={}",contextList.size(),JSON.toJSONString(contextList));
}
}
/**
* @description: 订单状态与场景的映射配置枚举类
* @Date : 2020/12/11 10:20 AM
* @Author : 石冬冬-Seig Heil
*/
enum OrderScene{
/**
* 秒批提交
*/
SECOND_BATCH_SUBMIT(status -> status.intValue() <= 1200, SceneEnum.SECOND_BATCH_SUBMIT),
/**
* 准入提交
*/
ADMITTANCE_SUBMIT(status -> status.intValue() > 1200 && status.intValue() <= 2100, SceneEnum.SECOND_BATCH_SUBMIT, SceneEnum.ADMITTANCE_SUBMIT),
/**
* 贷前提交
*/
LOAN_BEFORE_SUBMIT(status -> status.intValue() > 2100 && status.intValue() < 9999, SceneEnum.SECOND_BATCH_SUBMIT, SceneEnum.ADMITTANCE_SUBMIT, SceneEnum.BEFORE_LOAN_SUBMIT);
private Predicate<Integer> predicate;
private List<SceneEnum> sceneEnumList;
OrderScene(Predicate<Integer> predicate, SceneEnum... scenes){
this.predicate = predicate;
this.sceneEnumList = Arrays.stream(scenes).collect(Collectors.toList());
}
/**
* 根据状态获取场景
* @param status
* @return
*/
public static List<SceneEnum> getScene(Integer status){
Optional<OrderScene> optional = Stream.of(OrderScene.values()).filter(each -> each.predicate.test(status)).findFirst();
return optional.isPresent() ? optional.get().sceneEnumList : null;
}
}
/**
* @description: 包装请求参数
* @Date : 2020/12/11 10:19 AM
* @Author : 石冬冬-Seig Heil
*/
@Data
@NoArgsConstructor
public static class Request<T> {
/**
* 限制条数
*/
@ApiModelProperty(value = "限制条数",dataType = "java.lang.Integer")
protected int limit;
/**
* 指定范围
*/
@ApiModelProperty(value = "指定范围",dataType = "java.lang.List<T>")
protected List<T> scope;
}
}
/**
* @description: 车辆评估业务对象
* @Date : 2020/12/12 9:55 AM
* @Author : 石冬冬-Seig Heil
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class CarEvaluationBo {
/**
* car_price_evaluation_apply.id
*/
@ApiModelProperty(value="主键",name="id",dataType="java.lang.Integer")
private Integer id;
/**
* car_price_evaluation_apply.app_code
*/
@ApiModelProperty(value="订单号",name="app_code",dataType="java.lang.String")
private String appCode;
/**
* car_price_evaluation_apply.vin
*/
@ApiModelProperty(value="vin码",name="vin",dataType="java.lang.String")
private String vin;
/**
* car_price_evaluation_apply.evaluation_remarks
*/
@ApiModelProperty(value="评估备注",name="evaluation_remarks",dataType="java.lang.String")
private String evaluationRemarks;
}
3.1.11、RabbitConsumer
/**
* @description: rabbitmq消费者
* @Date : 2020/7/10 下午4:02
* @Author : 石冬冬-Seig Heil
*/
@Component
@Slf4j
public class RabbitConsumer {
final static SceneEnum SCENE_MERCHANT_ACCOUNT_SUBMIT = SceneEnum.MERCHANT_ACCOUNT_SUBMIT;
static String HOST_ADDRESS;
static {
try {
HOST_ADDRESS = InetAddress.getLocalHost().getHostAddress();
log.info("[RabbitConsumer],HOST_ADDRESS={}",HOST_ADDRESS);
} catch (UnknownHostException e) {
log.error("[RabbitConsumer],init HOST_ADDRESS exception",e);
HOST_ADDRESS = "127.0.0.0";
}
}
@Autowired
ExecutorService commonThreadPoolExecutor;
@Autowired
SubmitEventMulticaster submitEventMulticaster;
@Autowired
DiamondConfigProxy diamondConfigProxy;
@Autowired
RedisService redisService;
@Autowired
TraceLogFacade traceLogFacade;
/**
* 订阅订单中心变化
* @param message
*/
public void subscribeOrderStatus(String message){
final String LOG_TITLE = "subscribeOrderStatus#orderCenterStatus|ordercenter-key-node-message";
String appCode = null;
Integer status = null;
try {
BusMessage busMessage = JSON.parseObject(message,new TypeReference<BusMessage>(){});
log.info("{}, params={}", LOG_TITLE, JSON.toJSONString(busMessage));
appCode = busMessage.getAppCode();
status = busMessage.getStatus();
// 2. 查询是否在处理的状态范围
OrderStatusAttention orderStatusAttention = diamondConfigProxy.orderStatusAttention();
List<Integer> attentionStatusScope = orderStatusAttention.getScope();
if(status != null && !attentionStatusScope.contains(status)) {
log.debug("{}, 不需要关注状态 appCode={}, status={}", LOG_TITLE, appCode, status);
return;
}
SceneEnum scene = SceneEnum.valueOf(orderStatusAttention.getMapping().get(status));
SubmitEventContext context = SubmitEventContext.builder()
.appCode(appCode)
.scene(scene)
.build();
log.info("{} appCode={}, SubmitEventContext={}", LOG_TITLE, appCode, JSON.toJSONString(context));
//3、记录图谱记录
if(diamondConfigProxy.switchConfig().subscribeOrderStatus){
submitEventMulticaster.execute(context);
}
//4、记录链路日志
syncSaveTraceRecord(appCode,message,traceLog -> {
traceLog.setUrl("[" + HOST_ADDRESS + "]rabbitConsumer.subscribeOrderStatus");
traceLog.setResponseBody(JSONObject.toJSONString(context));
});
} catch (Exception e) {
log.error("[subscribeOrderStatus]异常,appCode={},message={}", appCode,JSONObject.toJSONString(message),e);
}
}
/**
* 订阅车商账号提交
* @param message
*/
public void subscribeMerchantAccountSubmit(String message){
final String LOG_TITLE = "subscribeOrderStatus#subscribeMerchantAccountSubmit|ordercenter-key-node-message";
Long dealerId = null;
try {
List<CarDealerInfoDTO> carDealerList = JSON.parseArray(message,CarDealerInfoDTO.class);
log.info("{}, params={}", LOG_TITLE, JSON.toJSONString(carDealerList));
if(CollectionsTools.isEmpty(carDealerList)){
log.warn("{}, params={}", LOG_TITLE, JSON.toJSONString(carDealerList));
return;
}
dealerId = carDealerList.get(0).getMerchantId();
SubmitEventContext context = SubmitEventContext.builder()
.merchantId(dealerId)
.carDealers(carDealerList)
.scene(SCENE_MERCHANT_ACCOUNT_SUBMIT)
.build();
log.info("{} merchantId={}, SubmitEventContext={}", LOG_TITLE, dealerId, JSON.toJSONString(context));
//1、记录图谱记录
if(diamondConfigProxy.switchConfig().subscribeMerchantAccountSubmit){
submitEventMulticaster.execute(context);
}
//2、记录链路日志
syncSaveTraceRecord(Objects.toString(dealerId),message,traceLog -> {
traceLog.setUrl("[" + HOST_ADDRESS + "]rabbitConsumer.subscribeMerchantAccountSubmit");
traceLog.setResponseBody(JSONObject.toJSONString(context));
});
} catch (Exception e) {
log.error("[subscribeMerchantAccountSubmit]异常,dealerId={},message={}", dealerId,message,e);
}
}
/**
* 记录链路日志
* @param appCode
* @param message
* @param caller
*/
void syncSaveTraceRecord(String appCode, String message, Consumer<TraceLog> caller){
TraceLog traceLog = TraceLog.builder()
.appCode(appCode)
.target(this.getClass().getPackage().getName() + "." + this.getClass().getSimpleName())
.requestBody(message)
.requestTime(TimeTools.createNowTime())
.responseTime(TimeTools.createNowTime())
.traceType(TraceTypeEnum.RABBIT_CONSUMER.getIndex()).build();
caller.accept(traceLog);
commonThreadPoolExecutor.execute(() -> traceLogFacade.saveRecord(traceLog));
}
@Data
static class BusMessage implements Serializable {
private Long messageId;
private String messageCode;
private String channel;
private String appCode;
//上一个状态
private Integer lastStatus;
//当前状态
private Integer status;
private String data;
private Date sendTime;
//初审增加征信类型
private Integer creditAuthType;
}
}
3.2、数据查重
上图是整个代码设计的UML类图,从上述图可以看出:
- 1、第一层的"API"层,对外提供HTTP接口。
- 2、第二层的"查询执行器",包装API层的请求,并交由查询处理器处理。
- 3、第三层的"查询处理器",抽象类仅派生一种业务场景多条件查询处理器
MultiSearchHandler
。 - 3、第四层的"命中查询处理器",按照业务场景派生出三个子类
PersonInfoHitQuerier
、VehicleInfoHitQuerier
、DealerInfoHitQuerier
,同时提供了两种数据命中模式ExactSearchMode
、SimilarSearchMode
。
3.2.1、MultiSearchRequest
/**
* @description: 多条件查重查询条件DTO
* @Date : 2020/12/15 3:17 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class MultiSearchRequest {
@ApiModelProperty(value="请求ID(可以使用UUID生成)",dataType="java.lang.String")
@NotNull(message = "请求ID[requestId]非空")
private String requestId;
@ApiModelProperty(value="查询条件",dataType="List<Condition>")
@NotNull(message = "查询条件[conditions]非空")
private List<Condition> conditions;
/**
* @description: 查询条件
* @Date : 2020/12/17 3:42 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public static class Condition{
@ApiModelProperty(value="查询字段类型",dataType="SourceTypeEnum")
private SourceTypeEnum sourceType;
@ApiModelProperty(value="查询字段名称",dataType="java.lang.String")
@NotNull(message = "查询字段名称[searchFieldName]不能为空!")
@NotEmpty(message = "查询字段名称[searchFieldName]不能为空!")
private String searchFieldName;
@ApiModelProperty(value="模糊查询分数区间",dataType="java.lang.Double")
@Size(max = 2, message = "查询字段名称[scoreRange]列表长度应该0-2")
private List<Double> scoreRange;
@ApiModelProperty(value="查询字段输入值",dataType="java.lang.String")
@NotNull(message = "查询字段输入值[searchFieldValue]不能为空!")
@NotEmpty(message = "查询字段输入值[searchFieldValue]不能为空!")
private String searchFieldValue;
@ApiModelProperty(value="查询字段描述",dataType="java.lang.String")
@NotNull(message = "查询字段描述[searchFieldDesc]不能为空!")
@NotEmpty(message = "查询字段描述[searchFieldDesc]不能为空!")
private String searchFieldDesc;
}
}
/**
* @description: 来源类型(1-主贷人;2-销售;3-销售主管;4-配偶;5-担保人;6-二手车卖方;7-紧急联系人1;8-紧急联系人2;9-车商法人;10-车商负责人;11-车商收款人;12-车辆;)
* @Date : 2020/11/27 4:08 PM
* @Author : 石冬冬-Seig Heil
*/
public enum SourceTypeEnum implements EnumValue {
CREDITOR_INFO(1,"主贷人"),
SELLER_INFO(2,"销售"),
SALES_MANAGER_INFO(3,"销售主管"),
MATE(4,"配偶"),
GUARANTOR_INFO(5,"担保人"),
SELLER_INFO_OF_OLD_CAR(6,"二手车卖方"),
EMERGENCY_CONTACT_ONE(7,"紧急联系人1"),
EMERGENCY_CONTACT_TWO(8,"紧急联系人2"),
CAR_DEALER_LEGAL_PERSON(9,"车商法人"),
CAR_DEALER_PRINCIPAL(10,"车商负责人"),
CAR_DEALER_PAYEE(11,"车商收款人"),
VEHICLE_INFO(12,"车辆"),
;
private int index;
private String value;
SourceTypeEnum(int index, String value ){
this.value = value;
this.index = index;
}
@Override
public int getIndex() {
return index;
}
@Override
public String getName() {
return value;
}
/**
* 根据索引获取对象
* @param index
* @return
*/
public static SourceTypeEnum getByIndex(int index){
return Stream.of(SourceTypeEnum.values()).filter(each -> each.getIndex() == index).findFirst().get();
}
/**
* 根据索引获取名称
* @param index
* @return
*/
public static String getNameByIndex(int index){
SourceTypeEnum find = Stream.of(SourceTypeEnum.values()).filter(each -> each.getIndex() == index).findFirst().get();
return null == find ? "" : find.getName();
}
}
3.2.2、SearchResultRe
/**
* @description: 查重结果对象
* @Date : 2020/12/15 3:29 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class SearchResultRe<E extends SearchResultRe.Record>{
@ApiModelProperty(value="提示信息",dataType="java.lang.String")
private String message;
@ApiModelProperty(value="查询字段名称",dataType="java.lang.String")
private String searchFieldName;
@ApiModelProperty(value="查询字段输入值",dataType="java.lang.String")
private String searchFieldValue;
@ApiModelProperty(value="查询字段描述",dataType="java.lang.String")
private String searchFieldDesc;
@ApiModelProperty(value="命中条数",dataType="java.lang.Integer")
private int hitCount;
@ApiModelProperty(value="命中记录",dataType="List<Record>")
private List<E> hitRecords;
/**
* 累加命中条数
* @param delta
*/
public synchronized void increaseHitCount(int delta){
this.hitCount += delta;
}
/**
* 累加命中记录
* @param hits
*/
public synchronized void addHits(List<E> hits){
if(!isEmpty(hits)){
if(isEmpty(this.hitRecords)){
this.hitRecords = new ArrayList<>(hits.size());
}
this.hitRecords.addAll(hits);
}
}
/**
* 判断集合元素是否为空
* @param collection
* @return
*/
boolean isEmpty(Collection<?> collection){
return null == collection || collection.isEmpty();
}
/**
* @description: 查重结果对象
* @Date : 2020/12/15 3:29 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@SuperBuilder
@NoArgsConstructor
@AllArgsConstructor
public static class Record{
@ApiModelProperty(value="主键ID",dataType="java.lang.Integer")
private Integer id;
@ApiModelProperty(value="外部数据id",dataType="java.lang.String")
private String externalId;
@ApiModelProperty(value="业务单号",dataType="java.lang.String")
private String appCode;
@ApiModelProperty(value="数据单号",dataType="java.lang.String")
private String dataCode;
/**
* {@link SceneEnum#getIndex()}
*/
@ApiModelProperty(value="场景",dataType="java.lang.Integer")
private Integer scene;
private String sceneDesc;
/**
* {@link SourceTypeEnum#getIndex()}
*/
@ApiModelProperty(value="来源类型",dataType="java.lang.Integer")
private Integer sourceType;
private String sourceTypeDesc;
}
/**
* @description: 自然人信息
* @Date : 2020/12/15 3:29 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@SuperBuilder
@NoArgsConstructor
public static class PersonInfoRecord extends Record{
@ApiModelProperty(value="姓名",dataType="java.lang.String")
private String name;
@ApiModelProperty(value="身份证号",dataType="java.lang.String")
private String idNo;
@ApiModelProperty(value="手机号",dataType="java.lang.String")
private String mobile;
@ApiModelProperty(value="银行卡号",dataType="java.lang.String")
private String creditCardNo;
@ApiModelProperty(value="省",dataType="java.lang.String")
private String provinceAddress;
@ApiModelProperty(value="市",dataType="java.lang.String")
private String cityAddress;
@ApiModelProperty(value="区",dataType="java.lang.String")
private String districtAddress;
@ApiModelProperty(value="详细地址",dataType="java.lang.String")
private String detailAddress;
}
/**
* @description: 车辆信息
* @Date : 2020/12/15 3:29 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@SuperBuilder
@NoArgsConstructor
public static class VehicleInfoRecord extends Record{
@ApiModelProperty(value="VIN码",dataType="java.lang.String")
private String vin;
@ApiModelProperty(value="公里数",dataType="java.lang.Integer")
private Integer mileage;
@ApiModelProperty(value="评估备注",dataType="java.lang.String")
private String evaluateRemark;
}
}
3.2.3、SearchController
/**
* @description: 查重数据
* @Date : 2020/12/15 4:03 PM
* @Author : 石冬冬-Seig Heil
*/
@RestController
@Api(description = "查重数据", tags = "查重数据")
@RequestMapping("/search")
public class SearchController {
@Autowired
SearchQueryExecutor searchQueryExecutor;
/**
* 多条件查询
* @return
*/
@PostMapping("/multiQuery")
@ApiOperation(value = "多条件查询", notes = "多条件查询")
@NoAuthRequired
@OvalValidator(value = "多条件查询[multiQuery]")
public Result<List<SearchResultRe>> multiQuery(@RequestBody MultiSearchRequest request) {
return searchQueryExecutor.execute(SearchQueryExecutor.Type.MULTI,request);
}
}
3.2.4、SearchQueryExecutor
/**
* @description: 查询执行器
* @Date : 2020/12/16 11:27 AM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class SearchQueryExecutor {
final Map<Type,AbstractSearchHandler> HANDLER_MAP = Maps.newHashMap();
@Resource
MultiSearchHandler multiSearchHandler;
@PostConstruct
void init(){
HANDLER_MAP.put(Type.MULTI, multiSearchHandler);
}
public enum Type{
/**
* 单项查询
*/
SIMPLE,
/**
* 多项
*/
MULTI
}
/**
* 执行
* @param type
* @param request
* @param <T>
*/
public <T> Result<List<SearchResultRe>> execute(Type type, T request){
if(Type.MULTI == type){
SearchQueryContext<MultiSearchRequest> context = SearchQueryContext.<MultiSearchRequest>builder().param((MultiSearchRequest)request).build();
HANDLER_MAP.get(type).execute(context);
return Result.suc(context.getResults());
}
return Result.suc();
}
}
3.2.5、SearchQueryContext
/**
* @description: 数据查重上下文对象
* @Date : 2020/12/16 11:06 AM
* @Author : 石冬冬-Seig Heil
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class SearchQueryContext<T> {
/**
* 请求ID
*/
private String requestId;
/**
* 查询条件
*/
private T param;
/**
* 查询结果
*/
private List<SearchResultRe> results;
/**
* 查询是否成功
*/
private boolean success;
/**
* 业务信息
*/
private String message;
public SearchQueryContext withSuccess(boolean success,String message){
this.setSuccess(success);
this.setMessage(message);
return this;
}
}
3.2.6、AbstractSearchHandler
/**
* @description: 抽象查询处理器
* @Date : 2020/12/16 11:08 AM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractSearchHandler<T> {
/**
* 上下文容器
*/
final ThreadLocal<SearchQueryContext<T>> CONTEXT = new ThreadLocal<>();
@Resource
DiamondConfigProxy diamondConfigProxy;
@Resource
HitQuerierManager hitQuerierManager;
@Resource
ExecutorService searchQueryPoolExecutor;
/**
* 外部调用执行方法
* @param context
*/
public void execute(SearchQueryContext<T> context){
try {
CONTEXT.set(context);
doQuery(context);
} catch (Exception e) {
log.error("[execute],requestId={},ctx={}",context.getRequestId(),JSONObject.toJSONString(context),e);
}finally {
CONTEXT.remove();
}
}
/**
* 执行查询
* @param context
*/
abstract void doQuery(SearchQueryContext<T> context);
/**
* 构建一个异常的查询结果
* @param condition
* @param message
* @return
*/
SearchResultRe buildExceptionSearchResult(MultiSearchRequest.Condition condition,String message){
return SearchResultRe.builder()
.searchFieldName(condition.getSearchFieldName())
.searchFieldValue(condition.getSearchFieldValue())
.searchFieldDesc(condition.getSearchFieldDesc())
.message(message)
.hitRecords(Collections.emptyList())
.build();
}
}
3.2.7、MultiSearchHandler
多条件查询处理器,这里采用多线程,以条件维度,以子线程处理,主线程阻塞等待所有子线程处理,并把命中数据结果统一封装返回。
/**
* @description: 抽象查询处理器
* @Date : 2020/12/16 11:08 AM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
@Component
public class MultiSearchHandler extends AbstractSearchHandler<MultiSearchRequest> {
@Override
void doQuery(SearchQueryContext<MultiSearchRequest> context) {
context.setRequestId(context.getParam().getRequestId());
String requestId = context.getRequestId();
//条件列表
List<MultiSearchRequest.Condition> conditions = context.getParam().getConditions();
//结果列表
List<SearchResultRe> results = Lists.newArrayListWithExpectedSize(conditions.size());
//Future任务列表
List<Future<HitQuerierContext>> futureList = new ArrayList<>(conditions.size());
for(MultiSearchRequest.Condition condition : conditions){
//参数验证
if(!checkCondition(condition)){
results.add(super.buildExceptionSearchResult(condition, MessageFormat.format("查询异常|参数非法,field={0}",condition.getSearchFieldName())));
continue;
}
Future<HitQuerierContext> future = searchQueryPoolExecutor.submit(() -> {
HitQuerierContext hitQuerierContext = HitQuerierContext.builder()
.requestId(requestId)
.condition(condition)
.build();
hitQuerierManager.execute(hitQuerierContext);
return hitQuerierContext;
});
futureList.add(future);
}
int count = 0;
for (Future<HitQuerierContext> f : futureList) {
try {
HitQuerierContext hitQuerierContext = f.get();
results.add(hitQuerierContext.getHitResult());
} catch (InterruptedException | ExecutionException e) {
log.error("[doQuery][InterruptedException|ExecutionException],requestId={},context={}",requestId, JSONObject.toJSON(context),e);
results.add(super.buildExceptionSearchResult(conditions.get(count), MessageFormat.format("查询异常|运行时异常,message={0}",e.getMessage())));
continue;
}
count++;
}
context.setResults(results);
context.withSuccess(Boolean.TRUE,"查询成功");
}
/**
* 参数验证
* @param condition
*/
boolean checkCondition(MultiSearchRequest.Condition condition){
Set<String> configFields = diamondConfigProxy.searchFieldConfig().keySet();
log.info("[checkCondition],configFields={}", configFields.toArray());
String searchFieldName = condition.getSearchFieldName();
return configFields.contains(searchFieldName);
}
}
3.2.8、HitQuerierManager
命中查询管理器,由于一个查询条件会去多张表进行数据查询,然后再数据合并,因此这里吧查询器统一注册给
hitList
,每个请求则统一调用execute
方法循环迭代所有查询器处理。
/**
* @description: 命中查询管理器
* @Date : 2020/12/17 4:21 PM
* @Author : 石冬冬-Seig Heil
*/
@Component
public class HitQuerierManager {
static final List<AbstractHitQuerier> hitList = new ArrayList<>();
@Resource
PersonInfoHitQuerier personInfoHitQuerier;
@Resource
DealerInfoHitQuerier dealerInfoHitQuerier;
@Resource
VehicleInfoHitQuerier vehicleInfoHitQuerier;
@PostConstruct
void init(){
hitList.add(personInfoHitQuerier);
hitList.add(dealerInfoHitQuerier);
hitList.add(vehicleInfoHitQuerier);
}
/**
* 执行查询
* @param context
*/
public void execute(HitQuerierContext context){
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = SearchResultRe.builder()
.searchFieldValue(condition.getSearchFieldValue())
.searchFieldName(condition.getSearchFieldName())
.searchFieldDesc(condition.getSearchFieldDesc())
.message("查询成功")
.build();
for(AbstractHitQuerier querier : hitList){
SearchResultRe hit = querier.execute(context);
hitResult.increaseHitCount(hit.getHitCount());
hitResult.addHits(hit.getHitRecords());
}
context.setHitResult(hitResult);
}
}
3.2.9、HitQuerierContext
/**
* @description: 数据查询器上下文
* @Date : 2020/12/17 3:38 PM
* @Author : 石冬冬-Seig Heil
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class HitQuerierContext {
/**
* 请求ID
*/
private String requestId;
/**
* 查询条件
*/
private MultiSearchRequest.Condition condition;
/**
* 查询结果
*/
private SearchResultRe hitResult;
}
3.2.10、AbstractHitQuerier
/**
* @description: 抽象命中查询器
* @Date : 2020/12/17 3:38 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractHitQuerier<E extends SearchResultRe.Record> {
/**
* 处理查询table
*/
protected final SearchFieldConfig.Table table;
@Resource
DiamondConfigProxy diamondConfigProxy;
public AbstractHitQuerier(SearchFieldConfig.Table table) {
this.table = table;
}
/**
* 查询
* @param context
* @return
*/
abstract SearchResultRe<E> doQuery(HitQuerierContext context);
/**
* 外部执行方法
* @param context
* @return
*/
public SearchResultRe execute(HitQuerierContext context){
SearchResultRe searchResultRe;
try {
if(!executeCurrent(context)){
searchResultRe = buildDefaultSearchResult(context.getCondition());
return searchResultRe;
}
searchResultRe = doQuery(context);
} catch (Exception e) {
log.error("[execute],requestId={},ctx={}",context.getRequestId(),JSONObject.toJSONString(context),e);
searchResultRe = buildDefaultSearchResult(context.getCondition());
searchResultRe.setMessage("查询异常|message=" + e.getMessage());
}
return searchResultRe;
}
/**
* 是否执行当前查询器
* @param context
* @return
*/
protected boolean executeCurrent(HitQuerierContext context){
return getSearchFieldConfig(context.getCondition().getSearchFieldName()).getTables().contains(table);
}
/**
* 构建默认查询结果
* @param condition
* @return
*/
SearchResultRe buildDefaultSearchResult(MultiSearchRequest.Condition condition){
String fieldName = condition.getSearchFieldName();
String fieldValue = condition.getSearchFieldValue();
SearchResultRe hitResult = SearchResultRe.builder()
.searchFieldName(fieldName)
.searchFieldValue(fieldValue)
.hitCount(0)
.hitRecords(Collections.emptyList())
.build();
return hitResult;
}
/**
* 获取配置字段
* @param fieldName
* @return
*/
SearchFieldConfig getSearchFieldConfig(String fieldName){
Map<String,SearchFieldConfig> mapping = diamondConfigProxy.searchFieldConfig();
SearchFieldConfig searchFieldConfig = mapping.get(fieldName);
log.info("[getSearchFieldConfig],fieldName={},config={}",fieldName, JSONObject.toJSONString(searchFieldConfig));
return searchFieldConfig;
}
/**
* 处理命中数据
* @param context
* @param queryForm
* @param recordsCaller
* @return
*/
List<E> hitRecords(HitQuerierContext context, Object queryForm, BiFunction<String,Object,List<E>> recordsCaller){
MultiSearchRequest.Condition condition = context.getCondition();
SearchFieldConfig searchFieldConfig = getSearchFieldConfig(condition.getSearchFieldName());
SearchFieldConfig.SearchMode searchMode = searchFieldConfig.getSearchMode();
return AbstractSearchMode.getAbstractSearchMode(searchMode).execute(searchFieldConfig, context, queryForm, recordsCaller);
}
}
/**
* @description: 车商信息命中查询
* @Date : 2020/12/17 3:51 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
@Component
public class DealerInfoHitQuerier extends AbstractHitQuerier<SearchResultRe.PersonInfoRecord> {
public DealerInfoHitQuerier() {
super(SearchFieldConfig.Table.DEALER_INFO);
}
@Resource
DealerInfoService dealerInfoService;
@Override
SearchResultRe<SearchResultRe.PersonInfoRecord> doQuery(HitQuerierContext context) {
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = super.buildDefaultSearchResult(condition);
DealerInfoForm queryForm = DealerInfoForm.builder().dataStatus(0).build();
List<SearchResultRe.PersonInfoRecord> hitRecords = super.hitRecords(context,queryForm,(String fieldName,Object searchForm) -> {
List<DealerInfo> recordList = dealerInfoService.queryList((DealerInfoForm)searchForm);
SearchFieldConfig config = getSearchFieldConfig(condition.getSearchFieldName());
if(SearchFieldConfig.SearchMode.SIMILAR == config.getSearchMode() && condition.getSearchFieldName().contains("Address")){
String provinceAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
String cityAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
String districtAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
String detailAddressFiledName = config.getMapping().get(config.getMapping().size()-1);
return BeanConverter.convertFromDealer(recordList,
each -> BeanTool.getObjectValue(each, provinceAddressFiledName),
each -> BeanTool.getObjectValue(each, cityAddressFiledName),
each -> BeanTool.getObjectValue(each, districtAddressFiledName),
each -> BeanTool.getObjectValue(each, detailAddressFiledName));
}
return BeanConverter.convertFromDealer(recordList,
null, null, null,null);
});
hitResult.setHitRecords(hitRecords);
hitResult.setHitCount(hitRecords.size());
return hitResult;
}
}
/**
* @description: 自然人信息命中查询
* @Date : 2020/12/17 3:51 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
@Component
public class PersonInfoHitQuerier extends AbstractHitQuerier<SearchResultRe.PersonInfoRecord> {
/**
* 手机号字段输入
*/
static final String MOBILE_FIELD = "mobile";
/**
* 地址类字段命名后缀
*/
static final String ADDRESS_FIELD_SUFFIX = "Address";
public PersonInfoHitQuerier() {
super(SearchFieldConfig.Table.PERSON_INFO);
}
@Resource
PersonInfoService personInfoService;
@Override
SearchResultRe<SearchResultRe.PersonInfoRecord> doQuery(HitQuerierContext context) {
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = super.buildDefaultSearchResult(condition);
PersonInfoForm queryForm = PersonInfoForm.builder().dataStatus(0).build();
List<SearchResultRe.PersonInfoRecord> hitRecords = super.hitRecords(context,queryForm,(String fieldName,Object searchForm) -> {
List<PersonInfo> recordList = personInfoService.queryList((PersonInfoForm)searchForm);
if(MOBILE_FIELD.equals(condition.getSearchFieldName())){
return BeanConverter.convertFromPerson(recordList, each -> BeanTool.getObjectValue(each, fieldName),
null, null, null, null);
}
SearchFieldConfig config = getSearchFieldConfig(condition.getSearchFieldName());
if(SearchFieldConfig.SearchMode.SIMILAR == config.getSearchMode() && condition.getSearchFieldName().contains(ADDRESS_FIELD_SUFFIX)){
String provinceAddressFiledName = config.getMapping().get(0);
String cityAddressFiledName = config.getMapping().get(1);
String districtAddressFiledName = config.getMapping().get(2);
String detailAddressFiledName = config.getMapping().get(3);
return BeanConverter.convertFromPerson(recordList, null,
each -> BeanTool.getObjectValue(each, provinceAddressFiledName),
each -> BeanTool.getObjectValue(each, cityAddressFiledName),
each -> BeanTool.getObjectValue(each, districtAddressFiledName),
each -> BeanTool.getObjectValue(each, detailAddressFiledName));
}
return BeanConverter.convertFromPerson(recordList);
});
hitResult.setHitRecords(hitRecords);
hitResult.setHitCount(hitRecords.size());
return hitResult;
}
}
/**
* @description: 车辆信息命中查询
* @Date : 2020/12/17 3:51 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
@Component
public class VehicleInfoHitQuerier extends AbstractHitQuerier<SearchResultRe.VehicleInfoRecord> {
public VehicleInfoHitQuerier() {
super(SearchFieldConfig.Table.VEHICLE_INFO);
}
@Resource
VehicleInfoService vehicleInfoService;
@Override
SearchResultRe<SearchResultRe.VehicleInfoRecord> doQuery(HitQuerierContext context) {
MultiSearchRequest.Condition condition = context.getCondition();
SearchResultRe hitResult = super.buildDefaultSearchResult(condition);
VehicleInfoForm queryForm = VehicleInfoForm.builder().dataStatus(0).build();
List<SearchResultRe.VehicleInfoRecord> hitRecords = super.hitRecords(context,queryForm,(String fieldName,Object searchForm) -> {
List<VehicleInfo> recordList = vehicleInfoService.queryList((VehicleInfoForm)searchForm);
return BeanConverter.convertFromVehicle(recordList);
});
hitResult.setHitRecords(hitRecords);
hitResult.setHitCount(hitRecords.size());
return hitResult;
}
}
3.2.11、AbstractSearchMode
/**
* @description: 命中查询方式
* @see com.creditease.horus.core.search.querier.mode.impl.ExactSearchMode
* @see com.creditease.horus.core.search.querier.mode.impl.SimilarSearchMode
* @Date : 2020/12/24 3:27 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
public abstract class AbstractSearchMode {
/**
* 策略集合
*/
public static Map<SearchFieldConfig.SearchMode, AbstractSearchMode> abstractSearchModeMap = new HashMap<>();
/**
* 选择策略
* @param modeEnum
* @return
*/
public static AbstractSearchMode getAbstractSearchMode(SearchFieldConfig.SearchMode modeEnum){
return abstractSearchModeMap.get(modeEnum);
}
/**
* 策略抽象方法
* @param searchFieldConfig
* @param context
* @param queryForm
* @param recordsCaller
* @param <E>
* @return
*/
public abstract <E extends SearchResultRe.Record> List<E> execute(SearchFieldConfig searchFieldConfig,
HitQuerierContext context,
Object queryForm,
BiFunction<String, Object, List<E>> recordsCaller);
/**
* 拷贝配置的扩展参数到实体类
* @param searchFieldConfig
* @param queryForm
*/
public void copyExclude (SearchFieldConfig searchFieldConfig, Object queryForm){
Map<String,Object> exclude = searchFieldConfig.getExclude();
if(MapUtils.isNotEmpty(exclude)){
BeanTool.copyFromOneMap(exclude,queryForm);
}
}
}
/**
* @description: 命中查询方式-精确处理方式
* @Date : 2020/12/24 3:28 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
@Component
public class ExactSearchMode extends AbstractSearchMode {
{
abstractSearchModeMap.put(SearchFieldConfig.SearchMode.EXACT, this);
}
/**
* 精准查询
* @param searchFieldConfig
* @param context
* @param queryForm
* @param recordsCaller
* @return
*/
@Override
public <E extends SearchResultRe.Record> List<E> execute(SearchFieldConfig searchFieldConfig,
HitQuerierContext context,
Object queryForm,
BiFunction<String, Object, List<E>> recordsCaller) {
//拷贝配置的扩展参数到实体类
copyExclude (searchFieldConfig, queryForm);
//入参条件
String requestId = context.getRequestId();
String fieldName = context.getCondition().getSearchFieldName();
String fieldValue = context.getCondition().getSearchFieldValue();
//结果集
List<E> hitRecords;
//有效查询参数
Map<String,Object> sourceValues = Maps.newHashMap();
//入参一个参数映射成查询两个参数(入参mobile对应数据库primaryMobile,SecondMobile)
List<String> mapping = searchFieldConfig.getMapping();
if(null != mapping && !mapping.isEmpty()){
hitRecords = Lists.newArrayList();
for(String fieldNameAlias : mapping){
sourceValues.put(fieldNameAlias,fieldValue);
//map键值对拷贝到实体类
BeanTool.copyFromOneMap(sourceValues,queryForm);
log.info("[execute][hitRecords],requestId={},queryForm={}",requestId, JSONObject.toJSONString(queryForm));
List<E> hitRecordsTemp = recordsCaller.apply(fieldNameAlias,queryForm);
log.info("[execute][hitRecords],requestId={},hitRecordsFromDB={}",requestId,hitRecordsTemp.size());
if(CollectionsTools.isNotEmpty(hitRecordsTemp)){
hitRecords.addAll(hitRecordsTemp);
}
//抹掉本次参数(本次参数设置为null,并拷贝到查询实体类)
sourceValues.put(fieldNameAlias,null);
}
}else{
sourceValues.put(fieldName,fieldValue);
BeanTool.copyFromOneMap(sourceValues,queryForm);
log.info("[execute][hitRecords],requestId={},queryForm={}",requestId,JSONObject.toJSONString(queryForm));
hitRecords = recordsCaller.apply(fieldName,queryForm);
log.info("[execute][hitRecords],requestId={},hitRecordsFromDB={}",requestId,hitRecords.size());
}
return hitRecords;
}
}
/**
* @description: 命中查询方式-相似度处理方式
* @Date : 2020/12/24 3:28 PM
* @Author : 石冬冬-Seig Heil
*/
@Slf4j
@Component
public class SimilarSearchMode extends AbstractSearchMode {
/**
* 相似度切分四级详细字段
*/
final String SIMILAR_DETAIL_FIELD = "detail";
/**
* 输出模型,地址字段名称
*/
final String OUT_MODEL_PROVINCE_ADDRESS = "provinceAddress";
final String OUT_MODEL_CITY_ADDRESS = "cityAddress";
final String OUT_MODEL_DISTRICT_ADDRESS = "districtAddress";
final String OUT_MODEL_DETAIL_ADDRESS = "detailAddress";
/**
* 相似度查询类型
*/
final String SIMILAR_TYPE = "m:organization.organization.name";
{
abstractSearchModeMap.put(SearchFieldConfig.SearchMode.SIMILAR, this);
}
/**
* 相似度查询
* @param context
* @param queryForm
* @param recordsCaller
* @return
*/
@Override
public <E extends SearchResultRe.Record> List<E> execute(SearchFieldConfig searchFieldConfig,
HitQuerierContext context,
Object queryForm,
BiFunction<String, Object, List<E>> recordsCaller) {
//拷贝配置的扩展参数到实体类
copyExclude (searchFieldConfig, queryForm);
//入参条件
String requestId = context.getRequestId();
String fieldName = context.getCondition().getSearchFieldName();
String fieldValue = context.getCondition().getSearchFieldValue();
//入参条件拆分(地址拆分为省、市、区、详细)
List<String> mapping = searchFieldConfig.getMapping();
String[] fieldValues = fieldValue.split("\\|");
//需要分词比较的详细地址(具体字段名称)
String similarField = mapping.get(mapping.size()-1);
//用户传过来的详细地址
String addressDetail = fieldValues[fieldValues.length-1];
HsmmAddressNormalizer anm = new HsmmAddressNormalizer();
String addressDetailFormat = fieldValues[0] + fieldValues[1] + fieldValues[2]
+ ( (HashMap<String,String>)anm.splitAddress(addressDetail) ).get(SIMILAR_DETAIL_FIELD);
//有效查询参数
Map<String,Object> sourceValues;
//结果集
List<E> hitRecords = Collections.emptyList();
if(CollectionUtils.isEmpty(mapping)){
log.warn("[execute]diamond mapping is null,requestId={},fieldName={}",requestId,fieldName);
return hitRecords;
}
sourceValues = Maps.newHashMap();
String[] mappingValues = mapping.toArray(new String[mapping.size()]);
//最后一项不作为查询条件
for (int i = 0; i < mappingValues.length - 1; i++) {
sourceValues.put(mappingValues[i],fieldValues[i]);
}
BeanTool.copyFromOneMap(sourceValues,queryForm);
log.info("[execute][hitRecords],requestId={},queryForm={}",requestId, JSONObject.toJSONString(queryForm));
List<E> hitRecordsTemp = recordsCaller.apply(fieldName,queryForm);
log.debug("[execute][hitRecords],requestId={},hitRecordsFromDB={}",requestId,hitRecordsTemp.size());
if(CollectionsTools.isEmpty(hitRecordsTemp)){
return hitRecords;
}
//匹配度分数区间,长度限制0-2【分数区间为空-没有分数要求;分数区间长度为1-最低分要求;分数区间长度为2-分数区间要求】
List<Double> scoreRangeList = context.getCondition().getScoreRange();
Double[] scoreRange = CollectionUtils.isEmpty(scoreRangeList) ? new Double[0] : scoreRangeList.toArray(new Double[scoreRangeList.size()]);
if(scoreRange.length == 0){
//分数区间为空-没有分数要求
return hitRecordsTemp;
}else{
String provinceAddressFromDb;
String cityAddressFromDb;
String districtAddressFromDb;
String detailAddressFromDb;
String detailAddressFromDbFormat;
Double score;
hitRecords = Lists.newArrayListWithExpectedSize(hitRecordsTemp.size());
for (E item : hitRecordsTemp) {
//相似分数符合条件的添加到结果集
provinceAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_PROVINCE_ADDRESS);
cityAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_CITY_ADDRESS);
districtAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_DISTRICT_ADDRESS);
detailAddressFromDb = BeanTool.getObjectValue(item, OUT_MODEL_DETAIL_ADDRESS);
detailAddressFromDbFormat = provinceAddressFromDb + cityAddressFromDb + districtAddressFromDb
+ ( (HashMap<String,String>)anm.splitAddress(detailAddressFromDb) ).get(SIMILAR_DETAIL_FIELD);
score = NLPUtil.getUtil().similarity(SIMILAR_TYPE, detailAddressFromDbFormat, addressDetailFormat);
log.info("[execute][hitRecords][similarScore],requestId={},addressDetailFormat={},addressDetailFromDb={},score={}",
requestId, addressDetailFormat, detailAddressFromDbFormat, score);
//分数区间长度为1-最低分要求
if(scoreRange.length == 1 && score >= scoreRange[0]){
hitRecords.add(item);
}
//分数区间长度为2-分数区间要求
if(scoreRange.length == 2 && score >= scoreRange[0] && score <= scoreRange[1]){
hitRecords.add(item);
}
}
}
return hitRecords;
}
}
4、扩展部分
4.1、查重服务请求参数
请求参数示例
{
"requestId":"1",
"conditions": [{
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "主贷人身份证号",
"searchFieldName": "idNo",
"searchFieldValue": "350****8114118"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "主贷人手机号",
"searchFieldName": "mobile",
"searchFieldValue": "186****2901"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "销售手机号",
"searchFieldName": "mobile",
"searchFieldValue": "182****4023"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "二手车卖方身份证号",
"searchFieldName": "idNo",
"searchFieldValue": "3522****2138"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "紧急联系人1手机号",
"searchFieldName": "mobile",
"searchFieldValue": "139****603"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "紧急联系人2手机号",
"searchFieldName": "mobile",
"searchFieldValue": "18****637"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "担保人身份证号",
"searchFieldName": "idNo",
"searchFieldValue": "350****38"
}, {
"sourceType": "CREDITOR_INFO",
"searchFieldDesc": "担保人手机号",
"searchFieldName": "mobile",
"searchFieldValue": "15****020"
}, {
"sourceType": "VEHICLE_INFO",
"searchFieldDesc": "车辆VIN",
"searchFieldName": "vin",
"searchFieldValue": "LFV****3721"
}]
}
响应结果示例
部分数据脱敏展示了,比如手机号、银行卡号、身份证号。
{
"code": 0,
"data": [
{
"hitCount": 1,
"hitRecords": [
{
"appCode": "F2009111915000180101",
"creditCardNo": "-",
"dataCode": "P20121700551635",
"externalId": "100021309",
"id": 243677,
"idNo": "350122**4118",
"mobile": "186**01",
"name": "郑**",
"scene": 1003,
"sourceType": 1
}
],
"message": "查询成功",
"searchFieldDesc": "主贷人身份证号",
"searchFieldName": "idNo",
"searchFieldValue": "3501**14118"
},
{
"hitCount": 2,
"hitRecords": [
{
"appCode": "F2009151915000180101",
"creditCardNo": "-",
"dataCode": "P20121700538028",
"externalId": "100022351",
"id": 219723,
"idNo": "-",
"mobile": "186**901",
"name": "郑**",
"scene": 1002,
"sourceType": 8
},
{
"appCode": "F2009111915000180101",
"creditCardNo": "-",
"dataCode": "P20121700551635",
"externalId": "100021309",
"id": 243677,
"idNo": "3501**118",
"mobile": "186**01",
"name": "郑**",
"scene": 1003,
"sourceType": 1
}
],
"message": "查询成功",
"searchFieldDesc": "主贷人手机号",
"searchFieldName": "mobile",
"searchFieldValue": "186**2901"
},
{
"hitCount": 1,
"hitRecords": [
{
"appCode": "F2009151915000180106",
"creditCardNo": "-",
"dataCode": "P20121700537447",
"externalId": "100022605",
"id": 218697,
"idNo": "3505**5550",
"mobile": "182**4023",
"name": "欧**",
"scene": 1003,
"sourceType": 1
}
],
"message": "查询成功",
"searchFieldDesc": "销售手机号",
"searchFieldName": "mobile",
"searchFieldValue": "182**023"
},
{
"hitCount": 6,
"hitRecords": [
{
"appCode": "F2011091915000180104",
"creditCardNo": "-",
"dataCode": "P20121700417419",
"externalId": "100038555",
"id": 7575,
"idNo": "35223**38",
"mobile": "-",
"name": "阮**",
"scene": 1002,
"sourceType": 6
},
//省略其他
],
"message": "查询成功",
"searchFieldDesc": "二手车卖方身份证号",
"searchFieldName": "idNo",
"searchFieldValue": "3522**32138"
},
{
"hitCount": 3,
"hitRecords": [
{
"appCode": "F2010191915000180107",
"creditCardNo": "-",
"dataCode": "P20121700481894",
"externalId": "100032121",
"id": 121031,
"idNo": "-",
"mobile": "139**603",
"name": "陈**",
"scene": 1002,
"sourceType": 7
},
//省略其他
],
"message": "查询成功",
"searchFieldDesc": "紧急联系人1手机号",
"searchFieldName": "mobile",
"searchFieldValue": "139**603"
},
{
"hitCount": 3,
"hitRecords": [
{
"appCode": "F2007281915000180103",
"creditCardNo": "-",
"dataCode": "P20121700447417",
"externalId": "100010141",
"id": 60345,
"idNo": "-",
"mobile": "1810**37",
"name": "林**",
"scene": 1002,
"sourceType": 8
},
//省略其他
],
"message": "查询成功",
"searchFieldDesc": "紧急联系人2手机号",
"searchFieldName": "mobile",
"searchFieldValue": "181**637"
},
{
"hitCount": 3,
"hitRecords": [
{
"appCode": "F2009281915000180104",
"creditCardNo": "-",
"dataCode": "P20121700442684",
"externalId": "100026661",
"id": 52033,
"idNo": "35012**938",
"mobile": "152**20",
"name": "许**",
"scene": 1003,
"sourceType": 1
},
//省略其他
],
"message": "查询成功",
"searchFieldDesc": "担保人身份证号",
"searchFieldName": "idNo",
"searchFieldValue": "35012****195938"
},
{
"hitCount": 4,
"hitRecords": [
{
"appCode": "F2009281915000180104",
"creditCardNo": "-",
"dataCode": "P20121700442684",
"externalId": "100026661",
"id": 52033,
"idNo": "3501**8",
"mobile": "152050**",
"name": "许**",
"scene": 1003,
"sourceType": 1
},
//省略其他
],
"message": "查询成功",
"searchFieldDesc": "担保人手机号",
"searchFieldName": "mobile",
"searchFieldValue": "152**"
},
{
"hitCount": 6,
"hitRecords": [
{
"appCode": "F2011091915000180104",
"dataCode": "V20121700417457",
"evaluateRemark": "正常。1.备胎槽照片重新拍摄,要求完整清晰。\n2.补充左右后叶子板流水槽照片\n3.补充左右前纵梁照片\n4.补充主副驾驶座椅滑轨照片\n5.有补领记录,补充车架拓印号照片(铁皮上的)",
"externalId": "100038555",
"id": 1045,
"mileage": 136029,
"scene": 1002,
"sourceType": 12,
"vin": "LFV4A24F7A30837**"
},
//省略其他
],
"message": "查询成功",
"searchFieldDesc": "车辆VIN",
"searchFieldName": "vin",
"searchFieldValue": "LFV4A24F7A3083**"
}
],
"msg": "操作成功",
"success": true
}
4.2、数据查重字段配置
为了提高数据查重接口的扩展性,基于配置化的元数据配置。
value包含如下参数
desc
:参数描述,无业务逻辑;仅仅作为字段说明使用。searchMode
:查询方式,目前仅支持两种EXACT
(精准查询)、SIMILAR
(相似度查询)。tables
:json字符串数组,适用于该域查询的表,目前表共三个(PERSON_INFO
、DEALER_INFO
、VEHICLE_INFO
)。mapping
:查询字段映射,字符串数据,查询形式以或作为条件,结果集会进行合并。exclude
:查询过滤条件,查询结果集以该配置参数作为过滤条件。字段key作为查询条件field,value作为条件。
{
"idNo": {
"desc": "身份证号",
"searchMode": "EXACT",
"mapping": [],
"tables": [
"PERSON_INFO",
"DEALER_INFO"
]
},
"name": {
"desc": "姓名",
"searchMode": "EXACT",
"mapping": [],
"tables": [
"PERSON_INFO",
"DEALER_INFO"
]
},
"mobile": {
"desc": "手机号(primaryMobile,SecondMobile)",
"searchMode": "EXACT",
"mapping": [
"primaryMobile",
"secondMobile"
],
"tables": [
"PERSON_INFO",
"DEALER_INFO"
],
"exclude": {
"sourceTypeScopeExclude": [
2,
3
]
}
},
"creditCardNo": {
"desc": "银行卡号",
"searchMode": "EXACT",
"mapping": [],
"tables": [
"PERSON_INFO",
"DEALER_INFO"
]
},
"companyAddress": {
"desc": "单位地址",
"searchMode": "SIMILAR",
"mapping": [
"companyAddressProvince",
"companyAddressCity",
"companyAddressDistrict",
"companyAddressDetail"
],
"tables": [
"PERSON_INFO",
"DEALER_INFO"
]
},
"censusAddress": {
"desc": "户籍地址",
"searchMode": "SIMILAR",
"mapping": [
"censusAddressProvince",
"censusAddressCity",
"censusAddressDistrict",
"censusAddressDetail"
],
"tables": [
"PERSON_INFO"
]
},
"residenceAddress": {
"desc": "居住地址",
"searchMode": "SIMILAR",
"mapping": [
"residenceAddressProvince",
"residenceAddressCity",
"residenceAddressDistrict",
"residenceAddressDetail"
],
"tables": [
"PERSON_INFO"
]
},
"vin": {
"desc": "车辆VIN",
"searchMode": "EXACT",
"mapping": [],
"tables": [
"VEHICLE_INFO"
]
}
}
5、总结
总体设计上运用了相关设计模式,并分成了多个模块,每个模块负责各自的业务逻辑职责。其中在数据查重接口设计上,考虑查询数据量比较多,基于输入的多个条件,运用并行处理,并把多个处理器的查询结果再进行合并,从而提高接口的性能。