eclipse+scala+spark+maven的配置

最新推荐文章于 2024-07-29 18:14:55 发布

DdouQiu

最新推荐文章于 2024-07-29 18:14:55 发布

阅读量2.8k

点赞数

分类专栏： spark 文章标签： eclipse spark scala maven

本文链接：https://blog.csdn.net/qlynh/article/details/74905921

版权

spark 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

1. 首先通过菜单的Help选中Eclipse Marketplace，出现如下界面：

再Find框中输入Scala，会出来Scala IDE插件，安装这个插件，就可以在eclipse上使用scala编写程序了。

2. 点击Help中的Install New Software，出来如下图，点击Add按钮添加url: http://alchim31.net/m2e-scala/update-site，name自己取。出来下图中的Maven Integration for Eclipse，选中所有到next步骤。

3.安装完成后，可去Help->Installation Details查看。

至此，eclipse可以用maven管理scala项目了。

4. 添加Spark依赖：

在pom.xml中添加依赖：

<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.1</version>
</dependency>

note: spark-core后面的2.11指的是scala的版本。

5. 错误：

在配置完eclipse+scala+maven的时候出现错误not found: type JUnitRunner，这个主要是因为缺少specs2-junit jar包引起的，在pom.xml中添加依赖：

<dependency>
<groupId>org.specs2</groupId>
<artifactId>specs2-junit_2.11</artifactId>
<version>3.8.9</version>
<scope>test</scope>
</dependency>

即可。

6.奉上我的pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.qiuqiu</groupId>
  <artifactId>scala</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>${project.artifactId}</name>
  <description>My wonderfull scala app</description>
  <inceptionYear>2015</inceptionYear>
  <licenses>
    <license>
      <name>My License</name>
      <url>http://....</url>
      <distribution>repo</distribution>
    </license>
  </licenses>

  <properties>
    <maven.compiler.source>1.6</maven.compiler.source>
    <maven.compiler.target>1.6</maven.compiler.target>
    <encoding>UTF-8</encoding>
    <scala.version>2.11.5</scala.version>
    <scala.compat.version>2.11</scala.compat.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11 -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.11</artifactId>
    <version>2.1.1</version>
   
</dependency>
    

    <!-- Test -->
  <!-- https://mvnrepository.com/artifact/junit/junit -->
		<dependency>
		    <groupId>junit</groupId>
		    <artifactId>junit</artifactId>
		    <version>4.8.1</version>
		    <scope>test</scope>
		</dependency>

   <!-- https://mvnrepository.com/artifact/org.specs2/specs2-core_2.11 -->
	<dependency>
	    <groupId>org.specs2</groupId>
	    <artifactId>specs2-core_2.11</artifactId>
	    <version>3.8.4</version>
	    <scope>test</scope>
	</dependency>
	<!-- https://mvnrepository.com/artifact/org.specs2/specs2-junit_2.11 -->
<dependency>
    <groupId>org.specs2</groupId>
    <artifactId>specs2-junit_2.11</artifactId>
    <version>3.8.9</version>
    <scope>test</scope>
</dependency>

   
    <dependency>
      <groupId>org.scalatest</groupId>
      <artifactId>scalatest_${scala.compat.version}</artifactId>
      <version>2.2.4</version>
      <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/junit/junit -->
	
    
  </dependencies>

  <build>
    <sourceDirectory>src/main/scala</sourceDirectory>
    <testSourceDirectory>src/test/scala</testSourceDirectory>
    <plugins>
      <plugin>
        <!-- see http://davidb.github.com/scala-maven-plugin -->
        <groupId>net.alchim31.maven</groupId>
        <artifactId>scala-maven-plugin</artifactId>
        <version>3.2.0</version>
        <executions>
          <execution>
            <goals>
              <goal>compile</goal>
              <goal>testCompile</goal>
            </goals>
            <configuration>
              <args>
                <arg>-make:transitive</arg>
                <arg>-dependencyfile</arg>
                <arg>${project.build.directory}/.scala_dependencies</arg>
              </args>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.18.1</version>
        <configuration>
          <useFile>false</useFile>
          <disableXmlReport>true</disableXmlReport>
          <!-- If you have classpath issue like NoDefClassError,... -->
          <!-- useManifestOnlyJar>false</useManifestOnlyJar -->
          <includes>
            <include>**/*Test.*</include>
            <include>**/*Suite.*</include>
          </includes>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>