- 博客(12)
- 收藏
- 关注
RSS订阅原创 业务功能1111
package cn.itcast.function;import cn.itcast.bean.CleanBean;import cn.itcast.bean.IndexBean;import cn.itcast.constant.Constant;import cn.itcast.util.DateUtil;import com.sun.org.apache.bcel.internal.generic.L2I;import org.apache.flink.streaming.api.fu
2021-10-22 00:23:57
76
原创 子业务1111
package cn.itcast.task;import cn.itcast.bean.CleanBean;import cn.itcast.config.QuotConfig;import cn.itcast.function.IndexSecListPutWindowFunction;import cn.itcast.function.IndexSecWindowFunction;import cn.itcast.function.KeyFunction;import cn.itcast
2021-10-22 00:23:02
66
原创 1 业务111
package cn.itcast.job;import cn.itcast.avro.AvroDeserializerSchema;import cn.itcast.avro.SseAvro;import cn.itcast.avro.SzseAvro;import cn.itcast.bean.CleanBean;import cn.itcast.config.QuotConfig;import cn.itcast.map.SseMap;import cn.itcast.map.Szse
2021-10-22 00:22:32
69
原创 function
package cn.itcast.streaming.function.window;import cn.itcast.entity.ItcastDataObj;import cn.itcast.entity.TripModel;import com.google.common.collect.Lists;import org.apache.flink.streaming.api.functions.windowing.RichWindowFunction;import org.apache.
2021-10-22 00:10:18
88
原创 watermark
package cn.itcast.streaming.watermark;import cn.itcast.entity.ItcastDataObj;import org.apache.flink.streaming.api.functions.*;import org.apache.flink.streaming.api.watermark.Watermark;import javax.annotation.Nullable;/*** 驾驶行程自定义水印对象，解决数据迟到30s的问题.
2021-10-22 00:08:41
682
原创 2021-10-22Task
package cn.itcast.streaming.task;import cn.itcast.entity.ItcastDataObj;import cn.itcast.streaming.sink.SrcDataToHBaseSink;import cn.itcast.streaming.sink.SrcDataToHBaseSinkOptimizer;import cn.itcast.streaming.sink.VehicleDetailSinkOptimizer;import cn
2021-10-22 00:08:11
660
原创 TripDriveTask
package cn.itcast.streaming.task;import cn.itcast.entity.ItcastDataObj;import cn.itcast.entity.TripModel;import cn.itcast.streaming.function.window.DriveSampleWindowFunction;import cn.itcast.streaming.function.window.DriveTripWindowFunction;import cn
2021-10-21 23:59:09
90
原创 BaseTask
package cn.itcast.streaming.task;import cn.itcast.utils.ConfigLoader;import org.apache.flink.api.common.restartstrategy.RestartStrategies;import org.apache.flink.api.common.serialization.DeserializationSchema;import org.apache.flink.api.common.seriali
2021-10-21 23:57:58
146
原创 完整代码1
/*** 需求：原始数据ETL操作* flink消费kafka数据，将消费出来的数据进行转换、清洗、过滤以后，正常的数据需要写入到hbase和hdfs，异常的数据写入到hdfs中* 1）正常数据写入hdfs和hbase* 2）异常数据写入到hbase*/public class KafkaSourceDataTask extends BaseTask { /*** 入口方法* @param args */ public st...
2021-10-21 23:50:06
122
原创 任务处理基类
/*** 定义所有task作业的父类，在父类中实现公共的代码* 加载配置文件内容到ParameterTool对象中* 1）flink流处理环境的初始化* 2）flink接入kafka数据源消费数据*/public abstract class BaseTask { //定义parameterTool工具类 public static ParameterTool parameterTool; public static String appName;...
2021-10-21 23:48:52
238
原创 消费kafka
package cn.itcast.streaming.task;import cn.itcast.entity.ItcastDataObj;import cn.itcast.streaming.sink.SrcDataToHBaseSink;import cn.itcast.streaming.sink.SrcDataToHBaseSinkOptimizer;import cn.itcast.streaming.sink.VehicleDetailSinkFunction;import cn.it...
2021-10-21 23:47:08
132
原创 Zyllink项目 实时ETL
1 pom<dependencies><!-- Flink依赖的java语言环境 --><dependency><groupId>org.apache.flink</groupId><artifactId>flink-java</artifactId><version>${flink.version}</version>&...
2021-10-21 22:33:01
76
空空如也
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人