IntelliJ IDEA 安装-Spark

 1、IntelliJ IDEA 下载地址

Other Versions - IntelliJ IDEA (jetbrains.com)https://www.jetbrains.com/idea/download/other.html

2、安装Scala

All Available Versions | The Scala Programming Language (scala-lang.org)https://www.scala-lang.org/download/all.html

3、IDEA安装Scala插件

Versions: Scala Plugin for IntelliJ IDEA & Android Studio | JetBrains Marketplacehttps://plugins.jetbrains.com/plugin/1347-scala/versions

4、IDEA配置Maven

Maven(apache-maven-3.6.3)安装目录->conf文件见，打开settings.xml文件，配置本地仓库

1.修改settings.xml文件

 <mirror>

      <id>alimaven</id>

      <name>aliyun maven</name>

   <url>http://maven.aliyun.com/nexus/content/groups/public/</url>

      <mirrorOf>central</mirrorOf>        

    </mirror>

2.IDEA配置Maven

3.IDEA创建Spark项目

4.添加Scala支持

5.添加依赖pom.xml

<dependencies>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.12</artifactId>
        <version>3.1.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>2.7.3</version>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>1.7.30</version>
    </dependency>

    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
    </dependency>
</dependencies>

6.添加日志文件

在src/main/resources下，新建log4j.properties文件

log4j.rootLogger=info,console

log4j.appender.console=org.apache.log4j.ConsoleAppender

log4j.appender.console.target=System.out

log4j.appender.console.layout=org.apache.log4j.PatternLayout

log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c %M(): %m%n

7.创建Scala源码包