最近公司这块需要开发一个简单的flink项目,在之前公司都是在现成的项目中添加代码,现在自己搭建一个项目,有些生疏了,记录一下搭建过程。
1.maven配置准备
首先完成配置文件设计 方便多环境配置切换
<profiles>
<profile>
<id>dev</id>
<properties>
<profiles.active>dev</profiles.active>
</properties>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
<profile>
<id>prod</id>
<properties>
<profiles.active>prod</profiles.active>
</properties>
</profile>
</profiles>
<build>
<filters>
<filter>src/main/prop/application-${profiles.active}.properties</filter>
</filters>
<resources>
<resource>
<directory>src/main/resources/</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
配置完验证一下 读取成功再进行后面的配置
2.依赖jar准备
<properties>
<scala.version>2.11</scala.version>
<flink.version>1.8.1</flink.version>
<fastjson.version>1.2.72</fastjson.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala_${scala.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_${scala.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-0.8_${scala.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
<dependency>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.11</version>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<version>2.5.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
</dependency>
</dependencies>
编译 打包scala插件配置
<build>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<sourceDir>src/main/scala</sourceDir>
<jvmArgs>
<jvmArg>-Xms64m</jvmArg>
<jvmArg>-Xmx1024m</jvmArg>
</jvmArgs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<filters>
<filter>src/main/prop/application-${profiles.active}.properties</filter>
</filters>
<resources>
<resource>
<directory>src/main/resources/</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
scala-sdk记得添加到idea中