一、配置文件配置
进入hadoop_home/etc/Hadoop目录
二、配置core_site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://xxxx:8020</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/spark/data/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>io.file.buffer.size</name>
<value>65535</value>
<description>Size of read/write buffer used in SequenceFiles.</description>
</property>
<property>
<name>io.native.lib.available</name>
<value>true</value>
<description>Should native hadoop libraries, if present, be used.</description>
</property>
<property>
<name>fs.hdfs.impl</name>
<value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
<description>The FileSystem for hdfs: uris.</description>
</property>
</configuration>
三、配置hadoop_env.sh
注意把里面的JAVA_HOME改为自已配的*注意要使用绝对路径
四、配置hdfs-site.xml
<configuration>
<property>
<name>dfs.http.address</name>
<value>xxxx:10075</value>
<description>
The address and the base port where the dfs namenode web ui will listen on.
If the port is 0 then the server will start on a free port.
</description>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>xxxx:10090</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/spark/data/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/spark/data/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.namenode.handler.count</name>
<value>100</value>
</property>
</configuration>
五、配置mapred-env.sh
注意把JAVA_HOME配置成自已的
六、配置slaves,即加入slave地址
七、配置yarn-env.sh
注意把JAVA_HOME配置成自已的
八、配置yarn-site.xml
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>xxxx</value>
</property>
</configuration>
八、分发文件夹到其他主机上
九、格式化集群
bin/hadoop namenode -format