目录
关键词
wordCount Demo运行报错关键词:
java.lang.NullPointerException
org.apache.spark.storage.BlockManagerMaster.registerBlockManager
java.io.InvalidClassException
org.apache.spark.rpc.netty.NettyRpcEndpointRef; local class incompatible: stream classdesc serialVersionUID = -4186747031772874359, local class serialVersionUID = 6257082371135760434
scala 版本
spark-streaming 版本
现象
Java端报错
19/11/05 15:06:05 INFO SparkEnv: Registering OutputCommitCoordinator
19/11/05 15:06:06 INFO Utils: Successfully started service 'SparkUI' on port 4040.
19/11/05 15:06:06 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://DD-HP5500:4040
19/11/05 15:06:06 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://axe1:7077...
19/11/05 15:06:06 INFO TransportClientFactory: Successfully created connection to axe1/192.168.86.101:7077 after 46 ms (0 ms spent in bootstraps)
19/11/05 15:06:26 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://axe1:7077...
19/11/05 15:06:46 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://axe1:7077...
19/11/05 15:07:06 ERROR StandaloneSchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up.
19/11/05 15:07:06 WARN StandaloneSchedulerBackend: Application ID is not initialized yet.
19/11/05 15:07:06 INFO SparkUI: Stopped Spark web UI at http://DD-HP5500:4040
19/11/05 15:07:06 INFO StandaloneSchedulerBackend: Shutting down all executors
19/11/05 15:07:06 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 56271.
19/11/05 15:07:06 INFO NettyBlockTransferService: Server created on DD-HP5500:56271
19/11/05 15:07:06 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
19/11/05 15:07:06 INFO CoarseGrainedSchedulerBackend$DriverEndpoint: Asking each executor to shut down
19/11/05 15:07:06 WARN StandaloneAppClient$ClientEndpoint: Drop UnregisterApplication(null) because has not yet connected to master
19/11/05 15:07:06 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/11/05 15:07:06 INFO MemoryStore: MemoryStore cleared
19/11/05 15:07:06 INFO BlockManager: BlockManager stopped
19/11/05 15:07:06 INFO BlockManagerMaster: BlockManagerMaster stopped
19/11/05 15:07:06 WARN MetricsSystem: Stoppi