SpringCloud DataFlow — 1. 自定义Processor

SpringCloud DataFlow — 0. 本地部署部署
SpringCloud DataFlow — 1. 自定义Processor
SpringCloud DataFlow — 2. 自定义Sink
SpringCloud DataFlow — 3. 暴露properties
SpringCloud DataFlow — 4. Prometheus + Grafana 监控
SpringCloud DataFlow — 5. 多分支负载

Processor

# 参考文档
https://dataflow.spring.io/docs/stream-developer-guides/streams/standalone-stream-kafka/#usagecostprocessor-processor

# Spring Initialzr
https://start.spring.io/

新建Maven项目

1. 基本信息

Group: etl.dmt.quick
Artiface: unpack-processor-kafka

2. 添加依赖

kafka | cloud stream | actuator | cloud connectors

3. 生成项目, 解压缩, 导入IDE

4. 创建UnpackProcessor.java

package etl.dmt.quick.unpackprocessorkafka;

import etl.dmt.quick.dto.InputDataDTO;
import etl.dmt.quick.dto.OutputDataDTO;

import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Processor;
import org.springframework.messaging.support.MessageBuilder;

@EnableBinding(Processor.class)
public class UnpackProcessor {
    private Processor processor;
    
    public CsvUnpack(Processor processor) {
        this.processor = processor;
    }
    
    @StreamListener(Processor.INPUT)
    public void unpack(InputDataDTO inputData) {
        OutputDataDTO outputData = new OutputDataDTO();
        for(int i=0; i<5; i++) {
            this.processor.output().send(MessageBuilder.withPayload(outputData).build());
        }
    }
}

4.1 也可以这样写input和output

package etl.dmt.quick.unpackprocessorkafka;

import etl.dmt.quick.dto.InputDataDTO;
import etl.dmt.quick.dto.OutputDataDTO;

import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Processor;
import org.springframework.messaging.handler.annotation.SendTo;


@EnableBinding(Processor.class)
public class UnpackProcessor {
    
    @StreamListener(Processor.INPUT)
    @SendTo(Processor.OUTPUT)
    public OutputDataDTO unpack(InputDataDTO inputData) {
        OutputDataDTO outputData = new OutputDataDTO();
        return outputData;
    }
}

5. 打包到本地maven仓库中

./mvnw clean install -Dmaven.test.skip=true