本文主要介绍如何在Centos7系统安装和配置单节点Hadoop3.0.0,并运用Hadoop MapReduce和Hadoop分布式文件系统(HDFS)执行wordcount测试样例。
资源和环境
jdk:jdk-8u11-linux-x64.rpm
hadoop:hadoop-3.0.0.tar.gz
操作系统:CentOS-7-x86_64
系统用户:root
安装jdk
[root@localhost zby]# rpm -ivh jdk-8u11-linux-x64.rpm
运行javac,出现如下画面表示安装成功
[root@localhost zby]# javac
Usage: javac <options> <source files>
where possible options include:
-g Generate all debugging info
-g:none Generate no debugging info
-g:{lines,vars,source} Generate only some debugging info
-nowarn Generate no warnings
-verbose Output messages about what the compiler is doing
此时jdk安装在/usr/java/jdk1.8.0_11下
设置环境变量,打开.bash_profile
[root@localhost java]# cd
[root@localhost ~]# vim .bash_profile
增加如下内容
export JAVA_HOME=/usr/java/jdk1.8.0_11
export PATH=$JAVA_HOME/bin:$PATH
执行source .bash_profile使配置生效
安装jps
执行jps命令提示bash: jps: command not found...,可通过以下方式解决
[root@localhost zby]# vim /etc/profile
添加export PATH="/usr/java/jdk1.8.0_11/bin:$PATH",保存退出,执行
[root@localhost zby]# source /etc/profile
[root@localhost zby]# jps
13049 Jps
安装和配置ssh免密码登录
[root@localhost zby]# yum install ssh
[root@localhost zby]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
97:d3:d3:62:e6:4b:7b:9a:c0:8f:b1:5d:01:e7:5c:4e root@localhost.localdomain
The key's randomart image is:
+--