增加节点。
复制一个虚拟机。
配置hosts文件,检查是否能ssh通。
Hadoop:
scp /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/slaves root@Worker1:/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/slaves
hadoop namenode –format/hdfs namenode -format
Spark:
scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves
root@Worker1:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves
出现问题:
只显示3个DataNode
http://www.cnblogs.com/wanqieddy/archive/2012/08/02/2619928.html
temp/data/里面的VERSION文件不同。导致有节点未能注册到Hadoop集群manager上!!!
scp 对拷文件夹 和 文件夹下的所有文件 对拷文件并重命名
对拷文件夹 (包括文件夹本身)
scp -r /home/wwwroot/www/charts/util root@192.168.1.65:/home/wwwroot/limesurvey_back/scp
对拷文件夹下所有文件 (不包括文件夹本身)
scp /home/wwwroot/www/charts/util/* root@192.168.1.65:/home/wwwroot/limesurvey_back/scp
对拷文件并重命名
scp /home/wwwroot/www/charts/util/a.txt root@192.168.1.65:/home/wwwroot/limesurvey_back/scp/b.text
修改slaves
root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker1:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves
slaves100% 899 0.9KB/s00:00
root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker2:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves
slaves100% 899 0.9KB/s00:00
root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker3:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves
slaves 100%899 0.9KB/s 00:00
root@Master:~# scp /usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves root@Worker4:/usr/local/spark/spark-1.6.0-bin-hadoop2.6/conf/slaves
slaves
启动时遇到问题:
full log in /usr/local/spark/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.master.Master-1-Master.out
报错master 内存不足。