【spark系列5】回归之LinearRegressionWithSGD

最新推荐文章于 2021-12-26 12:08:38 发布

sina微博_SNS程飞

最新推荐文章于 2021-12-26 12:08:38 发布

阅读量6.3k

点赞数

分类专栏：机器学习 spark 文章标签： spark LinearRegression

本文链接：https://blog.csdn.net/ksearch/article/details/24187851

版权

机器学习同时被 2 个专栏收录

33 篇文章 1 订阅

订阅专栏

spark

10 篇文章 0 订阅

订阅专栏

其它步骤请参考上一篇博客，转载请注明地址。

scala程序

import org.apache.spark.mllib.regression.LinearRegressionWithSGD
import org.apache.spark.mllib.regression.LabeledPoint

object ObLinearRegressionWithSGD{
    def run() {
        // Load and parse the data file
        val data = sc.textFile("D:/schoolar_tool/spark-0.9.1/mllib/data/ridge-data/lpsa.data")
        val parsedData = data.map { line =>
          val parts = line.split(',')
          LabeledPoint(parts(0).toDouble, parts(1).split(' ').map(x => x.toDouble).toArray)
        }

        // Building the model
        val numIterations = 20
        val model = LinearRegressionWithSGD.train(parsedData, numIterations)

        // Evaluate model on training examples and compute training error
        val valuesAndPreds = parsedData.map { point =>
          val prediction = model.predict(point.features)
          (point.label, prediction)
        }
        val MSE = valuesAndPreds.map{ case(v, p) => math.pow((v - p), 2)}.reduce(_ + _)/valuesAndPreds.count
        println("training Mean Squared Error = " + MSE)
    }
}