spark streaming demo
- spark streaming代码
val conf=new SparkConf().setMaster("local[*]").setAppName("sparkStreamingDemo")
...
需要注意的就是上面的master需要设置成local[*]。如果设置成local或者local[1]会报错如下:
WARN BlockManager: Block input-0-1437581601600 replicated to only 0 peer(s) instead of 1 peers
报错的原因参考该链接
将主要原因粘贴如下:
Do not run Spark Streaming programs locally with master configured as local or local[1]. This allocates only one CPU for tasks and if a receiver is running on it, there is no resource left to process the received data. Use at least local[2] to have more cores.
2.netcat命令
nc -l 7777