利用jiron数据开发平台提交flink任务到flink 集群,实现IDE控制台实时展示日志信息。
实现效果:
定义注解:
(开始和结束流程的切面)
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Inherited
public @interface ExecuteProcess {
ProcessType type();
}
(执行过程记录)
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Inherited
public @interface ProcessStep {
ProcessStepType type();
}
配置切面拦截注解。把执行的流程注册到上下文中,然后执行具体的操作,根据执行结果更新流程结果。
@Around(value = "@annotation(executeProcess)")
public Object processAround(ProceedingJoinPoint joinPoint, ExecuteProcess executeProcess) throws Throwable {
Object result;
Object processId = getProcessId(joinPoint);
String name = StrFormatter.format("{}/{}", executeProcess.type().getValue(), String.valueOf(processId));
ProcessType type = executeProcess.type();
//注册流程
contextHolder.registerProcess(type, name);
MDC.put(PROCESS_NAME, name);
try {
//处理
result = joinPoint.proceed();
//更新流程结果
contextHolder.finishedProcess(name, ProcessStatus.FINISHED, null);
} catch (Throwable e) {
//更新流程结果
contextHolder.finishedProcess(name, ProcessStatus.FAILED, e);
throw e;
} finally {
MDC.clear();
}
return result;
}
配置执行步骤切面进行日志更新
@Around(value = "@annotation(processStep)")
public Object processStepAround(ProceedingJoinPoint joinPoint, ProcessStep processStep) throws Throwable {
String processName = MDC.get(PROCESS_NAME);
if (TextUtils.isEmpty(processName)) {
log.warn(
"Process {} does not exist, This registration step {} was abandoned",
processName,
processStep.type());
return joinPoint.proceed();
}
Object result;
// Record the current step and restore it after the execution is completed
String parentStep = MDC.get(PROCESS_STEP);
ProcessStepType processStepType = processStep.type();
ProcessStepEntity step = contextHolder.registerProcessStep(processStepType, MDC.get(PROCESS_NAME), parentStep);
MDC.put(PROCESS_STEP, step.getKey());
contextHolder.appendLog(processName, step.getKey(), "Start Process Step:" + step.getType(), true);
try {
result = joinPoint.proceed();
contextHolder.finishedStep(MDC.get(PROCESS_NAME), step, ProcessStatus.FINISHED, null);
} catch (Exception e) {
contextHolder.finishedStep(MDC.get(PROCESS_NAME), step, ProcessStatus.FAILED, e);
throw e;
} finally {
// restored after the execution is complete
MDC.put(PROCESS_STEP, parentStep);
}
return result;
}
首先,提交任务,注册流程(日志:Start Process:FlinkSubmit/4)
@GetMapping("/submitTask")
@ApiOperation("Submit Task")
@Log(title = "Submit Task", businessType = BusinessType.SUBMIT)
//注册流程
@ExecuteProcess(type = ProcessType.FLINK_SUBMIT)
public Result<JobResult> submitTask(@ProcessId @RequestParam Integer id) throws Exception {
JobResult jobResult =
taskService.submitTask(TaskSubmitDto.builder().id(id).build());
if (jobResult.isSuccess()) {
return Result.succeed(jobResult, Status.EXECUTE_SUCCESS);
} else {
return Result.failed(jobResult, jobResult.getError());
}
}
(Start Process Step:SUBMIT_PRECHECK)
@ProcessStep(type = ProcessStepType.SUBMIT_PRECHECK)
public TaskDTO prepareTask(TaskSubmitDto submitDto) {
TaskDTO task = this.getTaskInfoById(submitDto.getId());
log.info("Start check and config task, task:{}", task.getName());
DinkyAssert.check(task);
if (StringUtils.isNotBlank(submitDto.getSavePointPath())) {
task.setSavePointStrategy(SavePointStrategy.CUSTOM.getValue());
task.setSavePointPath(submitDto.getSavePointPath());
}
task.setVariables(Optional.ofNullable(submitDto.getVariables()).orElse(new HashMap<>()));
return task;
}
........中间步骤省略
(Start Process Step:SUBMIT_EXECUTE)
@ProcessStep(type = ProcessStepType.SUBMIT_EXECUTE)
public JobResult executeJob(TaskDTO task) throws Exception {
JobResult jobResult = BaseTask.getTask(task).execute();
log.info("execute job finished,status is {}", jobResult.getStatus());
return jobResult;
}
前端通过api请求获取执行结果,在控制台展示
@GetMapping("/getProcess")
@ApiOperation("get process")
@ApiImplicitParam(name = "processName", value = "process name", dataType = "ProcessEntity")
public Result<ProcessEntity> listAllProcess(@RequestParam String processName) {
return Result.succeed(ConsoleContextHolder.getInstances().getProcess(processName));
}