使用VirtualBox安装Hadoop安装详解
一、前言
1、本文使用VirtualBox安装Hadoop,前期的centos7的配置请看:https://blog.csdn.net/qq_48395735/article/details/106816891
(按照博客做完第三步:使用moba连接虚拟机即可)
二、准备
相关安装包下载:链接:https://pan.baidu.com/s/1MOa9GqINlZF4avz0nVyGPg
提取码:w8uh
三、安装Hadoop
1、导入安装包、讲安装包拖入software文件包中
[root@hadoop101 opt]# cd /
[root@hadoop101 /]# mkdir software
2、下载解压缩工具
[root@hadoop101 softeare]# yum install -y unzip
3、解压缩安装包
[root@hadoop101 softeare]# tar -zxvf jdk-8u221-linux-x64.tar.gz -C /opt
[root@hadoop101 softeare]# tar -zxvf hadoop-2.6.0-cdh5.14.2.tar.gz
[root@hadoop101 softeare]# unzip apache-tomcat-8.5.45.zip /opt/tomcat8
4、为文件重名名,新建一个moba窗口2
cd /opt
[root@hadoop101 opt]# mv jdk-8u221-linux-x64/ jdk1.8.0_221
[root@hadoop101 opt]# mv hadoop-2.6.0-cdh5.14.2/ hadoop
5、配置java环境
vi /etc/profile
#添加以下内容
export JAVA_HOME=/opt/jdk1.8.0_221
export CLASSPATH=.:$JAVA_HOME/rt.jar:$JAVA_HOME/tools.jar:$JAVA_HOME/dt.jar
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin:$NODE_HOME/bin
6、查看java8环境是否配置成功
[root@hadoop101 opt]# source /etc/profile
[root@hadoop101 opt]# java -version
#出现以下代表java配置成功
java version "1.8.0_221"
Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)
7、修改主机名
[root@hadoop101 opt]# vi /etc/hostname
#修改为,看你自己取名
hadoop101
8、修改主机列表
[root@hadoop101 opt]# vi /etc/hosts
9、配置免密登陆,在窗口1执行命令
cd ~
ssh-keygen -t rsa -P ""
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
10、配置Hadoop
[root@hadoop101 hadoop]# cd /opt/hadoop/etc/hadoop
#此时是在etc下面的hadoop立马
[root@hadoop101 hadoop]# ll
11、配置hadoop-env.sh
#修改如下:
The java implementation to use.
export JAVA_HOME=/opt/jdk1.8.0_221
[root@localhost hadoop]# vi hadoop-env.sh
12、配置core-site.xml
[root@localhost hadoop]# vi core-site.xml
#修改如下
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://192.168.56.101:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop/tmp</value>
</property>
<property>
<name>hadoop.proxyuser.root.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.root.groups</name>
<value>*</value>
</property>
</configuration>
13、配置hdfs-site.xml
[root@localhost hadoop]# vi hdfs-size.xml
#修改如下
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop101:50090</value>
</property>
</configuration>
14、配置mapred-site.xml
[root@localhost hadoop]# mv mapred-site.xml.template mapred-site.xml
[root@localhost hadoop]# vi mapred-site.xm
[root@localhost hadoop]# vi mapred-site.xml
#修改如下:
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop101:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop101:19888</value>
</property>
</configuration>
15、配置yarn-site.xml
[root@localhost hadoop]# vi yarn-site.xml
#配置如下
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop101</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>604800</value>
</property>
</configuration>
16、hadoop环境变量配置
[root@localhost hadoop]# vi /etc/profile
#配置如下:shfit+g定位末尾,添加如下内容
export HADOOP_HOME=/opt/greeinstall/hadoop260
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
17、[root@localhost hadoop]# source /etc/profile
18、格式化HDFS
[root@localhost hadoop]# hadoop namenode -format
19、启动hadoop
[root@localhost hadoop]# start-all.sh
[root@localhost hadoop]# jps
20、在网页中输入http://192.168.56.101:50070
21:在网页中输入http://192.168.56.101:8088