【无标题】flink代码

从集合读取数据

// 定义样例类，传感器 id，时间戳，温度
case class SensorReading(id: String, timestamp: Long, temperature: Double)
object Sensor {
def main(args: Array[String]): Unit = {
val env = StreamExecutionEnvironment.getExecutionEnvironment
val stream1 = env
.fromCollection(List(
SensorReading(“sensor_1”, 1547718199, 35.8),
SensorReading(“sensor_6”, 1547718201, 15.4),
SensorReading(“sensor_7”, 1547718202, 6.7),
SensorReading(“sensor_10”, 1547718205, 38.1)
))
stream1.print(“stream1:”).setParallelism(1)
env.execute()
}
}

从文件读取数据

val stream2 = env.readTextFile(“YOUR_FILE_PATH”)

以 kafka 消息队列的数据作为来源

org.apache.flink flink-connector-kafka-0.11_2.12 1.10.1

具体代码如下：
val properties = new Properties()
properties.setProperty(“bootstrap.servers”, “localhost:9092”)
properties.setProperty(“group.id”, “consumer-group”)
properties.setProperty(“key.deserializer”,
“org.apache.kafka.common.serialization.StringDeserializer”)
properties.setProperty(“value.deserializer”,
“org.apache.kafka.common.serialization.StringDeserializer”)
properties.setProperty(“auto.offset.reset”, “latest”)
val stream3 = env.addSource(new FlinkKafkaConsumer011[String](“sensor”, new
SimpleStringSchema(), properties)

自定义 Soure

    除了以上的 source 数据来源，我们还可以自定义 source。需要做的，只是传入

一个 SourceFunction 就可以。具体调用如下：

 val stream4 = env.addSource( new MySensorSource())
 class MySensorSource extends SourceFunction[SensorReading]{
  // flag: 表示数据源是否还在正常运行
  var running: Boolean = true
  override def cancel(): Unit = {
     running = false
}
override def run(ctx: SourceFunction.SourceContext[SensorReading]): Unit
= {
       // 初始化一个随机数发生器
       val rand = new Random()
       var curTemp = 1.to(10).map(
       i => ( "sensor_" + i, 65 + rand.nextGaussian() * 20 )
       )
       var curTemp = 1.to(10).map(
       i => ( "sensor_" + i, 65 + rand.nextGaussian() * 20 )
       )
       Thread.sleep(100)
       }
       }
       }

Reduce

val stream2 = env.readTextFile(“YOUR_PATH\sensor.txt”)
.map( data => {
val dataArray = data.split(“,”)
SensorReading(dataArray(0).trim, dataArray(1).trim.toLong,
dataArray(2).trim.toDouble)
})
.keyBy(“id”)
.reduce( (x, y) => SensorReading(x.id, x.timestamp +1, y.temperature) )