- 安装nc,进行简单监听功能
- 命令yum install -y nc
- 安装成功后查看 nc -help 不报错即成功
- 启动Hadoop集群,spark集群,进入spark-shell界面,输入如下
import org.apache.spark.streaming.StreamingContext import org.apache.spark.streaming.StreamingContext._ import org.apache.spark.streaming.dstream.DStream import org.apache.spark.streaming.Duration import org.apache.spark.streaming.Seconds sc.setLogLevel("WARN") val ssc=new StreamingContext(sc,Seconds(5)) val lines=ssc.socketTextStream("Yang001",8888) val words=lines.flatMap(_.split(" ")) val wordCounts=words.map(x=>(x,1)).reduceByKey(_+_) wordCounts.print() ssc.start() ssc.awaitTermination()
在指定的主机IP执行 nc -lk 8888 随便输入字符串
5.接着出现大量error字段,
总结起来就这么个原因
6.解决方法,用自己的主机IP替换主机的hostname,即成功抛出结果
import org.apache.spark.streaming.StreamingContext
import org.apache.spark.streaming.StreamingContext._
import org.apache.spark.streaming.dstream.DStream
import org.apache.spark.streaming.Duration
import org.apache.spark.streaming.Seconds
sc.setLogLevel("WARN")
val ssc=new StreamingContext(sc,Seconds(5))
val lines=ssc.socketTextStream("192.168.168.215",8888)
val words=lines.flatMap(_.split(" "))
val wordCounts=words.map(x=>(x,1)).reduceByKey(_+_)
wordCounts.print()
ssc.start()
ssc.awaitTermination()
7.成功过后的界面
内心舒服多了,