1.设置静态ip
在CentOS桌面右上角的图标,右键修改
重启网卡,执行servicenetwork restart(/etc/rc.d/init.d/network restart)
验证:执行命令ifconfig
2.修改主机名
1.修改会话中的主机名,执行命令 hostname hadoop
2.修改配置文件中主机名,vi/etc/sysconfig/network 设置HOSTNAME=hadoop
验证:重启机器(hostname)
3.把hostname和ip绑定
执行命令 vi/etc/hosts,增加一行内容:192.168.80.20 Hadoop2,192.168.80.30 Hadoop3保存退出
验证:pinghadoop
4.关闭防火墙
执行命令 serviceiptables stop
验证:serviceiptables status
5.关闭防火墙的自动运行
执行命令chkconfig iptables off
chkconfig(查看服务chkconfig --list chkconfig --list |grep iptables)
验证:chkconfig--list | grep iptables
6.SSH(Secure Shell)免密码登录
1.执行命令:ssh-keygen-t rsa 产生密钥,位于~/.ssh文件夹中
2.执行命令 cp /home/hadoop/.ssh/id_rsa.pub/home/hadoop/.ssh/authorized_keys
(多台电脑免密码登录(不能用root账户登录) scp /home/hadoop/.ssh/id_rsa.pubhadoop@hadoop1:/home/hadoop/.ssh/authorized_keys)
authorized_keys应该包含集群所有的id_rsa.pub
验证:ssh hadoop2
7.安装jdk
1.执行命令 rm -rf/usr/local/* 删除/usr/local下的所有文件
2.使用winscp把jdk文件复制到 /usr/local/
3.执行命令 chmodu+x jdk-6u45-linux-x64.bin 赋予可执行权限
4.执行命令 ./jdk-6u45-linux-x64.bin 解压缩操作
5.执行命令 mvjdk-6u45-linux-x64.bin jdk 重命名
6.设置环境变量 vi/etc/profile 增加两行内容 export JAVA_HOME=/home/hadoop/soft/jdk export PATH=.:$JAVA_HOME/bin:$PATH
执行命令source /home/hadoop/.bash_profile 让设置立即生效
验证:java-version
8.安装Hadoop
1.执行命令 tar-zxvf hadoop-1.1.2.tar.gz 进行解压缩
2.执行命令 mv hadoop-1.1.2 hadoop 重命名
3.设置环境变量 vi /home/hadoop/.bash_profile增加一行内容 export HADOOP_HOME=/home/hadoop/soft/hadoop
修改一行内容 exportPATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH
执行命令source/home/hadoop/.bash_profile 让设置立即生效
4.修改hadoop的配置文件,位于$HADOOP_HOME/etc/hadoop/目录下,修改七个配置文件: hadoop-env.sh yarn-env.sh slaves core-site.xml
hdfs-site.xml mapred-site.xml yarn-site.xml
a.hadoop-env.sh
exportJAVA_HOME=/home/hadoop/soft/jdk/
b. yarn-env.sh
export JAVA_HOME=/home/hadoop/soft/jdk/
c.slaves
hadoop2
hadoop3
d.core-site.xml(file:/hamoe/hadoop/soft/hadoop/tmp(存放临时文件)目录需 要预先建立)
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop1:9000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/hamoe/hadoop/soft/hadoop/tmp</value>
</property>
<property>
<name>hadoop.proxyuser.hduser.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hduser.groups</name>
<value>*</value>
</property>
</configuration>
c.hdfs-site.xml {file:/hamoe/hadoop/soft/hadoop/name(存放元数据)
file:/home/hadoop/soft/hadoop/data(存放数据块) 目录需要预先建立}
<configuration>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop1:9001</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/hadoop/soft/hadoop/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/hadoop/soft/hadoop/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>
d.mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop1:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop1:19888</value>
</property>
</configuration>
e.yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>hadoop1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>hadoop1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>hadoop1:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>hadoop1:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>hadoop1:8088</value>
</property>
</configuration>
5.复制到其他的节点 scp-r /home/hadoop/soft/hadoop hadoop@192.168.80.20: /home/hadoop/soft/
6.执行命令 ./bin/hdfs namenode –format 对namenode进行格式化
7.start-all.sh 启动hadoop(jps 显示当前所有java进程pid)
8. 执行命令 ./sbin/start -dfs.sh 启动hdfs(jps 显示当前所有java进程pid)
此时在hadoop1上面运行的进程有:namenode,secondarynamenode
hadoop2和hadoop3上面运行的进程有:datanode
9. 执行命令 ./sbin/start -yarn.sh 启动year(jps 显示当前所有java进程pid)
此时在hadoop1上面运行的进程有:namenode,secondarynamenode,resourcemanager hadoop2和hadoop3上面运行的进程有:datanode,nodemanager