Spark问题

15/07/09 17:50:36 INFO SecurityManager: Changing view acls to: dev001,
15/07/09 17:50:36 INFO SecurityManager: Changing modify acls to: dev001,
15/07/09 17:50:36 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(dev001, ); users with modify permissions: Set(dev001, )
15/07/09 17:50:36 INFO Slf4jLogger: Slf4jLogger started
Exception in thread "main" org.jboss.netty.channel.ChannelException: Failed to bind to: /172.16.31.163:0
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:391)
	at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:388)
	at scala.util.Success$$anonfun$map$1.apply(Try.scala:206)
	at scala.util.Try$.apply(Try.scala:161)
	at scala.util.Success.map(Try.scala:206)
	at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
	at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
	at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.net.BindException: 无法指定被请求的地址
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:444)
	at sun.nio.ch.Net.bind(Net.java:436)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
	at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

相同问题：
  http://apache-spark-user-list.1001560.n3.nabble.com/org-jboss-netty-channel-ChannelException-Failed-to-bind-to-master-1xx-xx-xx-0-td8400.html
  stackoverflow上也有好多类似的



解决方法：

  1、某位外国人的结局方法 
 
 

  
    The problem is resolved.I have added SPARK_LOCAL_IP=master in both slaves also.When i changed this my slaves are working.
 
 
 
 

  
     Thank you all for your suggestions
 
 
  
  2、好像确实需要把worker和master的spark-env.sh文件配置相同的。如下方式会把worker和master配置完全复制的
     见如下：
       http://my.oschina.net/132722/blog/196027
     
 然后在修改${SPARK_HOME}/conf下的slaves文件，添加上所有工作节点的主机名或者ip，注意是一行一个，这些工作节点服务器也需要配置ssh无密码登录，并加入master的公匙到工作节点。当然可以不配置，但是每次启动都输入密码很纠结。 
 然后使用scp复制spark文件到其他工作节点。 
     
scp -r ${SPARK_HOME} hadoop@主机名:要复制到的目录
 
 

  
  

   
   
  
  
 
 
   然后就可以在master主机${SPARK_HOME}/bin目录下运行start-all.sh启动集群
  


==========================================
这个问题主要是出现在使用master来提交算法时出现的，当使用worker机器提交时不会出现。
也就是说master提交有问题然而worker提交就会出现问题。