Linux ubuntu-hadoop-2.9.2安装与配置

• 获取压缩包

从*https://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz*获得压缩包

$ wget https://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz

创建目录

$ sudo mkdir /opt/hadoop

解压文件

$ tar -xf hadoop-2.9.2.tar.gz

移动文件

$ sudo sudo mv hadoop-2.9.2 /opt/hadoop/

• 安装JDK

创建jvm目录

$ sudo mkdir /usr/lib/jvm

应该是不用预先创建的

$ sudo apt-get install  openjdk-8-jdk

检查JDK的安装

$ java -version

设置JAVA_HOME变量

$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/

检查JAVA_HOME

$ echo $JAVA_HOME

• 配置hadoop

其他配置方法若配置以后执行start-all.sh出现以下错误，通过本配置方法均可得到解决。

This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [0.0.0.0]
The authenticity of host ‘0.0.0.0 (0.0.0.0)’ can’t be established.
ECDSA key fingerprint is SHA256:SHA256_value
Are you sure you want to continue connecting (yes/no)? no
0.0.0.0: Host key verification failed.
starting yarn daemons
starting resourcemanager, logging to filename.out
localhost: Error: JAVA_HOME is not set and could not be found.

创建hadoop的软连接

$ sudo ln -s /opt/hadoop/hadoop-2.9.2/bin/hadoop /bin/hadoop

测试hadoop命令
有个错误：

/bin/hadoop: 行 27: /bin/…/libexec/hadoop-config.sh: 没有那个文件或目录

解决上述错误的办法是创建一个软连接

 $ sudo ln -s /opt/hadoop/hadoop-2.9.2/libexec/ /libexec

进入配置文件目录

$ cd /opt/hadoop/hadoop-2.9.2/etc/hadoop

配置hadoop-env.sh，core-site.xml，hdfs-site.xml
hadoop-env.sh

主要配置JAVA_HOME变量，如果直接使用$JAVA_HOME可能会出现找不到JAVA_HOME变量的错误。
core-site.xml

配置的时候注意路径，如果不是安装在/opt/hadoop下的相对应改变即可。
hdfs-site.xml
初始化配置

$ hadoop namenode -format

启动

$ ./start-dfs.sh

$ jps

为了方便也可以给start-dfs.sh在/bin下创建一个软连接，这样就不用进入安装目录执行了。