每天一点hadoop第六篇（搭建伪分布）

hadoop2.6.0伪分布搭建

搭建环境:centos6.5(64位虚拟机)
工具：编译好的hadoop2.6.0 JDK1.7（1.6编译不通过）
搭建过程：
1，设置静态ip:vi /etc/sysconfig/network-scripts/ifcfg-eth0
修改： BOOTPROTO=”static”
IPADDR=”192.168.253.22”
NETMASK=”255.255.255.0”
GATWAY=”192.168.253.22”
2.设置主机名：vi /etc/sysconfig/network
3.设置防火墙：
service iptables stop
chkconfig iptables off
service iptables status
chkconfig iptables –list
重启网卡：service network restart
4.安装JDK：
tar -zxvf jdk-7u79-linux-x64.tar.gz
设置环境变量：
export JAVA_HOME = /usr/java/jdk
export PATH = $PATH:$JAVA_HOME/bin
重启环境变量：source /etc/profile
java -version
5.设置ssh免登陆：
进入到要的陆客户的 ~/.ssh目录下
执行：ssh-keygen –t rsa(四个回车)
cat~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
6.解压:hadoop2.6.0
tar -zxvf hadoop2.6.0-tar.gz
7.修改环境变量：
vi /etc/profile

export HADOOP_INSTALL=/usr/opt/hadoop
export PATH=${HADOOP_INSTALL}/bin:${HADOOP_INSTALL}/sbin${PATH}
export HADOOP_MAPRED_HOME=${HADOOP_INSTALL}
export HADOOP_COMMON_HOME=${HADOOP_INSTALL}
export HADOOP_HDFS_HOME=${HADOOP_INSTALL}
export YARN_HOME=${HADOOP_INSTALLL}
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_INSTALL}/lib/natvie
export HADOOP_OPTS="-Djava.library.path=${HADOOP_INSTALL}/lib:${HADOOP_INSTALL}/lib/native"

8.修改配置文件：
1.修改：hadoop-env.sh
vi hadoop-env.sh (/usr/hadoop/hadoop/etc/hadoop)
export JAVA_HOME=/usr/java/jdk
2.修改：core –site.xml

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/opt/hadoop/tmp</value> <!--一定要配置 系统默认的缓存文件夹在每次重启后会被删除-->
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://master:9000</value>
</property>
</configuration>
3.修改：hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/usr/opt/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/opt/hadoop/dfs/data</value>
</property>
<property> <！--这个属性节点是为了防止后面eclipse存在拒绝读写设置的 -->
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>
4.修改：vi yarn-site.xml
`

mapreduce.framework.name
yarn

><property>  
><name>yarn.nodemanager.aux-services</name>  
><value>mapreduce_shuffle</value>  
></property>  
></configuration>`
>5.修改：vi mapred-site.xml
>`<property>
><name>mapred.job.tracker</name>
><value>master:9001</value>
></property>`

9.格式化:bin/hdfs namenode-format 或者hadoop namenode –format
10.启动:./sbin/start-dfs.sh ./sbin/start-yarn.sh
11.查看JPS

网页：http://192.168.253.22:50070 (HDFS管理界面)
http://192.168.253.22:8088 （MR管理界面）
参考：http://www.iyunv.com/thread-38384-1-1.html
http://www.centoscn.com/hadoop/2015/0118/4525.html