1、IntelliJ IDEA 下载地址
Other Versions - IntelliJ IDEA (jetbrains.com)https://www.jetbrains.com/idea/download/other.html
2、安装Scala
3、IDEA安装Scala插件
4、IDEA配置Maven
Maven(apache-maven-3.6.3)安装目录->conf文件见,打开settings.xml文件,配置本地仓库
1.修改settings.xml文件
<mirror>
<id>alimaven</id>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
2.IDEA配置Maven
3.IDEA创建Spark项目
4.添加Scala支持
5.添加依赖pom.xml
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>3.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.7.3</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.30</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
6.添加日志文件
在src/main/resources下,新建log4j.properties文件
log4j.rootLogger=info,console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c %M(): %m%n