idea开发flink程序在windows上运行

flink程序使用java和scala语言都可以开发，java使用的人多，下面就用实现在idea上开发java的flink程序。

 

1.下载flink运行程序

flink-1.9.0-bin-scala_2.12.tgz

下载地址：

1）https://download.csdn.net/download/water17ball/14975857

2）https://www.apache.org/dyn/closer.lua/flink/flink-1.9.0/flink-1.9.0-bin-scala_2.12.tgz

下载解压后即可使用。 

其中：目录中的内容如下：

D:\diskD\programs\flink-1.9.0>dir

2021/01/31  11:21    <DIR>          .
2021/01/31  11:21    <DIR>          ..
2021/01/31  11:21    <DIR>          bin
2021/01/31  11:21    <DIR>          conf
2021/01/31  11:21    <DIR>          examples
2021/01/31  11:21    <DIR>          lib
2019/08/20  00:19            11,357 LICENSE
2021/01/31  11:21    <DIR>          licenses
2021/01/31  14:34    <DIR>          log
2019/08/20  00:21           794,639 NOTICE
2021/01/31  11:21    <DIR>          opt
2021/01/31  11:21    <DIR>          plugins
2019/08/20  00:19             1,308 README.txt

运行 Flink 需要安装 Java 7.x 或更高的版本，需要提前安装好，在dos命令符下面敲java可以正常使用。

 

进入bin目录，

运行start-cluster.bat启动flink服务：

D:\diskD\programs\flink-1.9.0\bin>start-cluster.bat
Starting a local cluster with one JobManager process and one TaskManager process.
You can terminate the processes via CTRL-C in the spawned shell windows.
Web interface by default on http://localhost:8081/.

最后一行显示flink的web界面地址，复制下来。

在浏览器中输入http://localhost:8081/即可访问。

服务启动起来了，接下来开始idea的程序编辑。

 

2. idea开发flink程序

2.1新建maven项目：

点击：File -> New -> Project,按照下面选择：

然后下一步，输入自定义的maven的坐标信息：

然后下一步，配置maven的信息：

然后下一步，点击finish完成创建。

 

2.2 引入flink的jar包

由于flink程序使用flink包中的类，需要引入flink的jar包。

进入解压的flink目录的lib目录：

然后，在idea中选择jar包：

File -> Project Structure, 然后点击左侧的library，再点击小+号，选择上面的jar包，进行添加。

这样就添加完成：

 

2.3编写flink程序：

 遵循是什么、为什么、怎么做的方针，我们不会教大家编写flink程序，只是让大家认识到flink程序是什么。

于是，我们从flink的官方例子中，找到单词统计的jar包，解压后反序列化一下，得到代码。

package com.test;

import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;

public class WordCountData {
    public static final String[] WORDS = new String[]{"To be, or not to be,--that is the question:--", "Whether 'tis nobler in the mind to suffer", "The slings and arrows of outrageous fortune", "Or to take arms against a sea of troubles,", "And by opposing end them?--To die,--to sleep,--", "No more; and by a sleep to say we end", "The heartache, and the thousand natural shocks", "That flesh is heir to,--'tis a consummation", "Devoutly to be wish'd. To die,--to sleep;--", "To sleep! perchance to dream:--ay, there's the rub;", "For in that sleep of death what dreams may come,", "When we have shuffled off this mortal coil,", "Must give us pause: there's the respect", "That makes calamity of so long life;", "For who would bear the whips and scorns of time,", "The oppressor's wrong, the proud man's contumely,", "The pangs of despis'd love, the law's delay,", "The insolence of office, and the spurns", "That patient merit of the unworthy takes,", "When he himself might his quietus make", "With a bare bodkin? who would these fardels bear,", "To grunt and sweat under a weary life,", "But that the dread of something after death,--", "The undiscover'd country, from whose bourn", "No traveller returns,--puzzles the will,", "And makes us rather bear those ills we have", "Than fly to others that we know not of?", "Thus conscience does make cowards of us all;", "And thus the native hue of resolution", "Is sicklied o'er with the pale cast of thought;", "And enterprises of great pith and moment,", "With this regard, their currents turn awry,", "And lose the name of action.--Soft you now!", "The fair Ophelia!--Nymph, in thy orisons", "Be all my sins remember'd."};

    public WordCountData() {
    }

    public static DataSet<String> getDefaultTextLineDataSet(ExecutionEnvironment env) {
        return env.fromElements(WORDS);
    }
}

 

package com.test;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.utils.ParameterTool;
import org.apache.flink.util.Collector;

public class WordCount {
    public WordCount() {
    }

    public static void main(String[] args) throws Exception {
        ParameterTool params = ParameterTool.fromArgs(args);
        ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
        env.getConfig().setGlobalJobParameters(params);
        Object text;
        if (params.has("input")) {
            text = env.readTextFile(params.get("input"));
        } else {
            System.out.println("Executing WordCount example with default input data set.");
            System.out.println("Use --input to specify file input.");
            text = WordCountData.getDefaultTextLineDataSet(env);
        }

        DataSet<Tuple2<String, Integer>> counts = ((DataSet)text).flatMap(new WordCount.Tokenizer()).groupBy(new int[]{0}).sum(1);
        if (params.has("output")) {
            counts.writeAsCsv(params.get("output"), "\n", " ");
            env.execute("WordCount Example");
        } else {
            System.out.println("Printing result to stdout. Use --output to specify output path.");
            counts.print();
        }

    }

    public static final class Tokenizer implements FlatMapFunction<String, Tuple2<String, Integer>> {
        public Tokenizer() {
        }

        public void flatMap(String value, Collector<Tuple2<String, Integer>> out) {
            String[] tokens = value.toLowerCase().split("\\W+");
            String[] var4 = tokens;
            int var5 = tokens.length;

            for(int var6 = 0; var6 < var5; ++var6) {
                String token = var4[var6];
                if (token.length() > 0) {
                    out.collect(new Tuple2(token, 1));
                }
            }

        }
    }
}

在java目录下面新建这两个类，同时将自带的App类删除。

点击14行位置的绿色三角，启动main程序。

D:\diskD\java\jdk1.8.0_271\bin\java.exe ... ...

Executing WordCount example with default input data set.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.
(bourn,1)
(coil,1)
(come,1)
(d,4)
(dread,1)
(is,3)
(long,1)
(make,2)
(more,1)
(must,1)
(no,2)
(oppressor,1)
(pangs,1)
(perchance,1)
(sicklied,1)
(something,1)
(takes,1)
(these,1)
(us,3)
(what,1)
(with,3)
(a,5)
(contumely,1)
(die,2)
(flesh,1)
(himself,1)
(his,1)
(hue,1)
(in,3)
(mind,1)
(now,1)
(or,2)
(orisons,1)
(rather,1)
(regard,1)
(resolution,1)
(rub,1)
(so,1)
(sweat,1)
(thought,1)
(undiscover,1)
(wish,1)
(you,1)
(against,1)
(arms,1)
(awry,1)
(bodkin,1)
(end,2)
(heartache,1)
(insolence,1)
(love,1)
(might,1)
(moment,1)
(mortal,1)
(name,1)
(not,2)
(office,1)
(pale,1)
(puzzles,1)
(s,5)
(sea,1)
(the,22)
(those,1)
(thousand,1)
(thus,2)
(troubles,1)
(unworthy,1)
(but,1)
(devoutly,1)
(dream,1)
(enterprises,1)
(fair,1)
(fly,1)
(for,2)
(fortune,1)
(grunt,1)
(life,2)
(may,1)
(others,1)
(remember,1)
(respect,1)
(spurns,1)
(take,1)
(to,15)
(we,4)
(weary,1)
(when,2)
(whips,1)
(wrong,1)
(after,1)
(and,12)
(arrows,1)
(ay,1)
(be,4)
(cast,1)
(country,1)
(er,1)
(great,1)
(know,1)
(merit,1)
(my,1)
(nobler,1)
(of,15)
(off,1)
(pith,1)
(say,1)
(scorns,1)
(shocks,1)
(this,2)
(thy,1)
(traveller,1)
(turn,1)
(death,2)
(delay,1)
(does,1)
(dreams,1)
(ills,1)
(quietus,1)
(sleep,5)
(slings,1)
(suffer,1)
(time,1)
(under,1)
(who,2)
(whose,1)
(action,1)
(bare,1)
(calamity,1)
(despis,1)
(he,1)
(law,1)
(makes,2)
(nymph,1)
(o,1)
(ophelia,1)
(opposing,1)
(outrageous,1)
(proud,1)
(returns,1)
(shuffled,1)
(than,1)
(them,1)
(tis,2)
(whether,1)
(all,2)
(bear,3)
(by,2)
(conscience,1)
(consummation,1)
(cowards,1)
(currents,1)
(fardels,1)
(from,1)
(give,1)
(have,2)
(heir,1)
(lose,1)
(man,1)
(native,1)
(natural,1)
(patient,1)
(pause,1)
(question,1)
(sins,1)
(soft,1)
(that,7)
(their,1)
(there,2)
(will,1)
(would,2)

Process finished with exit code 0

3.打包上传运行

将上面的程序打成jar包。

在flink管理界面上面，点击上传，即可运行。

即可查看到执行结果：

 

 

参考文献：

1.在 Windows 上安装和运行 Flink1.9  https://blog.csdn.net/zhanaolu4821/article/details/104547831