一、问题描述
使用sparkml库的linalg类库等,尽管引入spark mllib依赖,但是发现无法导入相应库,运行后报错:
Error:(2, 25) object ml is not a member of package org.apache.spark
import org.apache.spark.ml.classification.LogisticRegression
Error:(4, 25) object ml is not a member of package org.apache.spark
import org.apache.spark.ml.feature.VectorAssembler
Error:(5, 25) object ml is not a member of package org.apache.spark
import org.apache.spark.ml.linalg
Error:(6, 25) object ml is not a member of package org.apache.spark
import org.apache.spark.ml.regression.LinearRegression
Error:(46, 13) not found: value model
val s = model.summary.totalIterations
二、问题原因
排除了很多原因:
(1)已经导入spark的maven依赖
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.1.0</version>
<!-- <scope>runtime</scope>-->
</dependency>
(2)查看maven依赖,确实有这个jar包,以及对应类库
(3)最后发现竟然因为是没有导入源码,而导致无法引入,坑啊!
三、解决问题
1.右键maven的pom依赖
2.导入源码如下图
3.然后就可以导入相应类库了
import org.apache.spark.ml.classification.LogisticRegression
import org.apache.spark.ml.feature.VectorAssembler
import org.apache.spark.ml.linalg
import org.apache.spark.ml.regression.LinearRegression