今天先做入门,本人也是第一天学习,看以后的时间,教程也有可能太监
> 从下至上架构层次
安装和部署
jdk 我用的是 1.8 scala2.11 安装步骤就不赘述了
1.windows安装flink
> 下载安装Flink
https://www.apache.org/dyn/closer.lua/flink/flink-1.6.4/flink-1.6.4-bin-hadoop26-scala_2.11.tgz
> 在本地解压 进入bin目录
>执行start-cluster.bat
> 访问http://localhost:8081
2.socket 方式 简单使用flink
1.下载netcat。下载地址:https://eternallybored.org/misc/netcat/
2.解压文件夹‘
3 打开dos 窗口 验证是否通信可以成功
服务器端命令:
nc.exe -l -p [端口号]
客户端命令示例:
nc 192.168.35.13 9999
连接成功
注意 nc这个客户端 只能可以一个客户端连接,
如果代码要连接的话 就不能再有dos窗口连接了
原因没有细致研究
如果启动nc的时候报以下错误
需要关闭windows 安全中心病毒防护
3.编写scala程序 可以直接上flink官网
1.直接运行 官网的jar 包.\bin\flink run .\examples\streaming\SocketWindowWordCount.jar --port 9999
2.将官网的jar包放到自己的idea执行
object SocketWindowWordCount2 {
def main(args: Array[String]) : Unit = {
// the port to connect to
val port: Int = 9999
// get the execution environment
val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
// get input data by connecting to the socket
val text = env.socketTextStream("192.168.31.174", port, '\n')
println("--------------------")
// parse the data, group it, window it, and aggregate the counts
val windowCounts = text
.flatMap { w => w.split("\\s") }
.map { w => WordWithCount(w, 1) }
.keyBy("word")
.timeWindow(Time.seconds(5) ,Time.seconds(1))
.sum("count")
// print the results with a single thread, rather than in parallel
windowCounts.print().setParallelism(1)
env.execute("Socket Window WordCount")
}
// Data type for words with count
case class WordWithCount(word: String, count: Long)
}
public class SocketWindowWordCountJava {
public static void main(String[] args) throws Exception {
// 定义链接接口
final int port=9999;
// 得到执行环境对象
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// 链接socket之后,读取data
DataStream<String> text = env.socketTextStream("localhost", port, "\n");
// parse the data, group it, window it, and aggregate the counts
DataStream<WordWithCount> windowCounts = text
.flatMap(new FlatMapFunction<String, WordWithCount>() {
public void flatMap(String value, Collector<WordWithCount> out) {
for (String word : value.split("\\s")) {
out.collect(new WordWithCount(word, 1L));
}
}
})
.keyBy("word")
.timeWindow(Time.seconds(5), Time.seconds(1))
.reduce(new ReduceFunction<WordWithCount>() {
public WordWithCount reduce(WordWithCount a, WordWithCount b) {
return new WordWithCount(a.word, a.count + b.count);
}
});
// print the results with a single thread, rather than in parallel
windowCounts.print().setParallelism(1);
env.execute("Socket Window WordCount");
}
// Data type for words with count
public static class WordWithCount {
public String word;
public long count;
public WordWithCount() {}
public WordWithCount(String word, long count) {
this.word = word;
this.count = count;
}
@Override
public String toString() {
return word + " : " + count;
}
}
}
启动nc服务端,运行代码
4 打成jar包 jar包方式运行
启动flink
制定要执行的类名 和 jar包的路径
在nc命令输入文字
问题
按照正常情况,在下面这个路径下会把内容打出来,不知道是不是windows版本的问题没有输出,有时间安装个linux版本再试试