hadoop-2.6.0-cdh5.7.0 HDFS 伪分布式安装

最新推荐文章于 2020-11-21 21:26:32 发布

风继续吹-小海

最新推荐文章于 2020-11-21 21:26:32 发布

阅读量232

点赞数

分类专栏： hadoop

本文链接：https://blog.csdn.net/qq_29416019/article/details/82796941

版权

7 篇文章 0 订阅

订阅专栏

查看官方文档相关要求：
http://archive.cloudera.com/ 搜索需要的版本如果需要下载加上 .tar.gz 就可以

Required software for Linux include:

Java™ must be installed. Recommended Java versions are described at HadoopJavaVersions.
ssh must be installed and sshd must be running to use the Hadoop scripts that manage remote Hadoop daemons

1.1 安装与hadoop版本相符合的java版本
mkdir /usr/java
tar zxvf jdk-7u80-linux-x64.tar.gz -C /usr/java

vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.7.0_80
export PATH= $JAVA_HOME/bin:$ PATH

查看java版本:
java -versions

1.2 查看sshd服务是否运行

service sshd status

正式安装：

1.添加sudo权限的无密码访问的hadoop用户
useradd -r hadoop -d /home/hadoop
vi /etc/sudoers
hadoop ALL=(ALL) NOPASSWD: ALL

新建目录
mkdir /home/hadoop/app -p

解压软件：
tar zxvf hadoop-2.6.0-cdh5.7.0.tar.gz

npack the downloaded Hadoop distribution. In the distribution, edit the file etc/hadoop/hadoop-env.sh to define some parameters as follows:

export JAVA_HOME=/usr/java/jdk1.7.0_80
export HADOOP_PREFIX=/home/hadoop/app/hadoop

try the following command:

$ bin/hadoop

Now you are ready to start your Hadoop cluster in one of the three supported modes:

Local (Standalone) Mode 单机没有进程
Pseudo-Distributed Mode 伪分布式 1台机器进程
Fully-Distributed Mode 分布式进程生产

configuration

Use the following:

etc/hadoop/core-site.xml:

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop01:9000</value>
    </property>
</configuration>

etc/hadoop/hdfs-site.xml:

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

$ ssh-keygen -t dsa -P ‘’ -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
cd .ssh
chmod 600 authorized_keys

export JAVA_HOME=/usr/java/jdk1.7.0_80
export HADOOP_PREFIX=/home/hadoop/app/hadoop
export PATH= $HADOOP_PREFIX/bin:$ JAVA_HOME/bin:$PATH

The following instructions are to run a MapReduce job locally. If you want to execute a job on YARN, see YARN on Single Node.

Format the filesystem: 格式化
$ bin/hdfs namenode -format
Start NameNode daemon and DataNode daemon:
$ sbin/start-dfs.sh

关注