Hadoop伪分布式集群搭建

 

Ubuntu 操作系统

 

1  & vim etc/hosts

2 配置jdk环境变量

& vi /etc/profile

 

保存退出

激活 & source /etc/profile

验证java环境变量 java –version

二 配置hadoop

&  vi /etc/profile

export JAVA_HOME=/root/package/jdk       

export HADOOP_HOME=/root/package/Hadoop

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

vi /etc/selinux/config

把SELINUX=enforcing 改成 SELINUX=disabled

chmod 777 /root/package/jdk/bin/javac  为javac配置权限

chmod 777  /root/package/hadoop/bin/Hadoop

 

chmod 777  /root/package/jdk/bin/java

 

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

配置文件

& vi Hadoop-env.sh

export JAVA_HOME=/root/package/jdk       

export HADOOP_COMMON_HOME=/root/package/hadoop

& vi core-site.xml

<property>

  <name>fs.defaultFS</name>

  <value>hdfs://master:9000</value>

<property>

 

<property>

  <name>Hadoop.tmp.dir</name>

  <value>/root/package/Hadoop/tmp</value>

<property>

 

 

 

 

 

 

hdfs-site.xml

<configuration>

<property>

  <name>dfs.replication</name>

  <value>2</value>

</property>

 

<property>

  <name>dfs.namenode.secondary.http-address</name>

  <value>192.168.1.10:50090</value>

</property>

 

</configuration>

 

 

 

 

 

mapreduce.-site.xml

<property>

  <name>mapreduce.framework.name</name>

  <value>yarn</value>

</property>

 

yarn-site.xml

<property>

  <name>yarn.resourcemanager.hostname</name>

  <value>master</value>

</property>

<property>

  <name>yarn.nodemanager.aux-services</name>

  <value>mapreduce_shuffle</value>

</property>

 

在package下执行

sudo chown -R root:root hadoop/*   为hadoop赋权

 

 

 

 

启动HDFS为分布式模式：格式化namenode。执行

& hdfs namenode -format  (注意-为英文，大坑)

启动hdfs：

  & start-all.sh

输入jps查看启动状态

 & jps

       在浏览器中输入http://localhost:50070/

输入 http://localhost:8088/