hadoop或者spark 增加节点

增加节点。

复制一个虚拟机。

配置hosts文件，检查是否能ssh通。

 

Hadoop:

  
  
scp /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/slaves root@Worker1:/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/slaves

格式化：

   
   
hadoop namenode –format/hdfs namenode -format

 

Spark:

  
  
scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker1:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves

 

出现问题：

只显示3个DataNode

http://www.cnblogs.com/wanqieddy/archive/2012/08/02/2619928.html

 

temp/data/里面的VERSION文件不同。导致有节点未能注册到Hadoop集群manager上！！！

scp 对拷文件夹 和 文件夹下的所有文件 对拷文件并重命名

 

对拷文件夹 (包括文件夹本身)

scp -r   /home/wwwroot/www/charts/util root@192.168.1.65:/home/wwwroot/limesurvey_back/scp

对拷文件夹下所有文件 (不包括文件夹本身)

scp   /home/wwwroot/www/charts/util/* root@192.168.1.65:/home/wwwroot/limesurvey_back/scp

 

 对拷文件并重命名

scp   /home/wwwroot/www/charts/util/a.txt root@192.168.1.65:/home/wwwroot/limesurvey_back/scp/b.text

 

修改slaves

 

root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker1:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves

slaves100%  899     0.9KB/s00:00   

root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker2:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves

slaves100%  899     0.9KB/s00:00   

root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker3:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves

slaves                                        100%899     0.9KB/s   00:00

root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker4:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves

slaves

 

 

 

 

启动时遇到问题：

full log in /usr/local/spark/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.master.Master-1-Master.out

 

报错master 内存不足。