java判断读到末尾_Flink实战：自定义KafkaDeserializationSchema(Java/Scala)

微信公众号： 大数据开发运维架构

关注可了解更多大数据相关的资讯。问题或建议，请公众号留言;

如果您觉得“大数据开发运维架构”对你有帮助，欢迎转发朋友圈

---

kafka中的数据通常是键值对的，所以我们这里自定义反序列化类从kafka中消费键值对的消息，为方便大家学习，这里我实现了Java/Scala两个版本，由于比较简单这里直接上代码：

一、Scala代码：

1.自定义反序列化类：

package comhadoop.ljs.flink010.kafkaimport org.apache.flink.api.common.typeinfo.{TypeHint, TypeInformation}import org.apache.flink.streaming.connectors.kafka.KafkaDeserializationSchemaimport org.apache.kafka.clients.consumer.ConsumerRecord/**  * @author: Created By lujisen  * @company ChinaUnicom Software JiNan  * @date: 2020-04-25 18:31  * @version: v1.0  * @description: comhadoop.ljs.flink010.kafka  */class MyKafkaDeserializationSchema  extends KafkaDeserializationSchema[ConsumerRecord[String, String]]{  /*是否流结束，比如读到一个key为end的字符串结束，这里不再判断，直接返回false 不结束*/  override def isEndOfStream(t: ConsumerRecord[String, String]): Boolean ={    false  }  override def deserialize(record: ConsumerRecord[Array[Byte], Array[Byte]]): ConsumerRecord[String, String] = {    new ConsumerRecord(record.topic(),record.partition(),record.offset(),new String(record.key(),"UTF-8"),new String(record.value(),"UTF-8"))  }  /*用于获取反序列化对象的类型*/  override def getProducedType: TypeInformation[ConsumerRecord[String, String]] = {    TypeInformation.of(new TypeHint[ConsumerRecord[String, String]] {})  }}

2.主函数类：

package comhadoop.ljs.flink010.kafkaimport java.util.Propertiesimport org.apache.flink.api.common.functions.MapFunctionimport org.apache.flink.streaming.api.datastream.DataStreamimport org.apache.flink.streaming.api.environment.StreamExecutionEnvironmentimport org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerimport org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionimport org.apache.kafka.clients.consumer.ConsumerRecordimport org.apache.kafka.common.serialization.StringDeserializer/**  * @author: Created By lujisen  * @company ChinaUnicom Software JiNan  * @date: 2020-04-25 16:32  * @version: v1.0  * @description: comhadoop.ljs.fl