Hadoop集群搭建

Hadoop集群搭建

一、Linux系统配置

1、在所有的机器上都建立相同的目录，也可以就建立相同的用户，以该用户的home路径来做hadoop的安装路径。例如在所有的机器上都建立了名为fan的用户：/home/fan。

etc/hosts文件末尾添加IP-主机映射关系

2、rysnc

vi/etc/rysnc.conf

rsync--daemon

客户端：

rsync -avzP  mjiang@venus::hadoop  hadoop-1.0.0/etc/hadoop/

2、ssh 免密码登录

http://blog.csdn.net/j3smile/article/details/7259963

附录1：

#!/bin/bash
filename=/home/mjiang/.ssh/authorized_keys
slaves=(mars jupiter saturn)
for i in ${slaves[@]}
do
    echo $i
    scp $filename  mjiang@$i:$filename
done

3、安装hadoop，借助rysnc

tar zxvf hadoop-1.0.0-bin.tar.gz

4、安装jdk，借助rysnc

su tar zxvf jdk-7u1-linux-i586.tar.gz -C /usr/share/

5、配置文件修改

http://hadoop.apache.org/common/docs/r0.19.2/cn/cluster_setup.html

vi hadoop-1.0.0/etc/hadoop/hadoop-env.sh

export JAVA_HOME=/usr/share/jdk1.7.0_01

chmod+x hadoop-1.0.0/sbin/*

6、开始

./sbin/stop-all.sh

stoppingjobtracker

mars: stoppingtasktracker

jupiter:stopping tasktracker

saturn:stopping tasktracker

no namenodeto stop

（如果出现了这个那么就会出现

mjiang@venus~/hadoop-1.0.0 $ hadoop fs -ls /

12/02/1516:31:32 INFO ipc.Client: Retrying connect to server:venus/211.83.105.12:9000. Already tried 0 time(s).）

mars: stoppingdatanode

jupiter:stopping datanode

saturn:stopping datanode

venus:stopping secondarynamenode

hadoopnamenode -format

12/02/1516:39:13 INFO common.Storage: Storage directory/tmp/hadoop-mjiang/dfs/name has been successfully formatted.

./sbin/start-dfs.sh

..12/02/1516:39:13 INFO common.Storage: Storage directory/tmp/hadoop-mjiang/dfs/name has been successfully formatted....

./sbin/start-mapred.sh

在客户端：

mjiang@mars~ $ hadoop fs -mkdir /user/mjiang

mkdir: cannotcreate directory /user/mjiang: File exists

－－－是同步的