二、开发并运行flink程序
1. 创建package和object(略)
2. 编写程序
package code.book.idetes
import org.apache.flink.api.scala._
object WordCountJob {
def main(args: Array[String]) {
val env = ExecutionEnvironment.getExecutionEnvironment
val text = env.readTextFile("hdfs://qingcheng11:9000/input/flink/README.txt")
val counts = text.flatMap(_.toLowerCase.split("\\W+"))
.map((_, 1)).groupBy(0).sum(1)
counts.print()
}
}
3.运行效果
三、打包到服务器上运行
1.在pom.xml的中配置打包插件
<project>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>code.book.idetes.WordCountJob</mainClass>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
</manifest>
</archive>
<classesDirectory>
</classesDirectory>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
2.打包程序
如果顺利的话,在进行package操作后,将会生成。simple-flink-1.0-SNAPSHOT.jar文件
此文件的名称跟我们在maven的pom.xml配置是有关系的。因为在pom.xml中配置了
<artifactId>simple-flink</artifactId>
<version>1.0-SNAPSHOT</version>
所以产生了文件simple-flink-1.0-SNAPSHOT.jar
3.上传到服务器,并运行jar
${FLINK_HOME}/bin/flink run -m qingcheng11:6123 simple-flink-1.0-SNAPSHOT.jar