python调用sparkmlib_python – 结合Spark Streaming MLlib

我试图使用随机森林模型来预测一组示例,但似乎我不能使用该模型对示例进行分类.

这是pyspark中使用的代码：

sc = SparkContext(appName="App")

model = RandomForest.trainClassifier(trainingData, numClasses=2, categoricalFeaturesInfo={}, impurity='gini', numTrees=150)

ssc = StreamingContext(sc, 1)

lines = ssc.socketTextStream(hostname, int(port))

parsedLines = lines.map(parse)

parsedLines.pprint()

predictions = parsedLines.map(lambda event: model.predict(event.features))

和在集群中编译时返回的错误：

Error : "It appears that you are attempting to reference SparkContext from a broadcast "

Exception: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transformation. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063.

有没有办法使用从静态数据生成的模型来预测流式示例？

谢谢你们,我真的很感激!!!!