在这里插入代码片
Hadoop安装
注:hadoop下载地址:https://archive.apache.org/dist/hadoop/core/hadoop-3.3.0/
Jdk下载地址:https://www.oracle.com/java/technologies/javase-jdk14-downloads.html
安装hadoop
此次安装为hadoop3.3和jdk1.8,以ubuntu20.04系统为例:
- 先准备2台服务器,一台为master,一台为slave,
- 将hadoop和jdk放到/opt下
- 将两台服务器用test01用户登录,并赋予root权限
在/etc//sudoers文件里修改 - Master可以无密码输入登录slave、localhost服务器中
#sudo apt-get install openssh-server
#ssh-keygen -t rsa -P ‘’ -f ~/.ssh/id_rsa
#cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
#ssh-copy-id test01@IP - 解压hadoop和jdk压缩包
#sudo tar -zxvf hadoop-3.3.0.tar.gz
#sudo tar -zxvf jdk-8u261-linux-x64.tar.gz - 设置环境变量
#sudo vim /etc/profile
添加:export HADOOP_HOME=/opt/hadoop-3.3.0
export PATH= H A D O O P H O M E / b i n : HADOOP_HOME/bin: HADOOPHOME/bin:HADOOP_HOME/sbin:$PATH - 修改配置参数
cd /opt/hadoop-3.3.0/etc/hadoop/
#sudo vim hadoop-env.sh
取消注释并配置jdk地址:export JAVA_HOME=/opt/jdk1.8.0_261
#sudo vim hdfs-site.xml
添加参数:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
> <property>
> <name>dfs.namenode.name.dir</name>
> <value>file:/opt/hadoop-3.3.0/tmp/dfs/name</value>
> </property>
> <property>
> <name>dfs.datanode.data.dir</name>
> <value>file:/opt/hadoop-3.3.0/tmp/dfs/data</value>
> </property>
#sudo vim core-site.xml
添加参数:
<property>
<name>fs.defaultFS</name>
<value>hdfs://解析地址:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/opt/hadoop-3.3.0/tmp</value>
</property>
#sudo vim mapred-site.xml
添加参数
:<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
#sudo vim yarn-site.xml
添加参数:
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master域名</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
#sudo vim workers
添加节点
8. 将core-site.xml;hadoop-env.sh;hdfs-site.xml;mapred-site.xml;yarn-site.xml;workers文件替换次节点文件
9. 启动hadoop
$ cd /opt/hadoop-3.3.0/sbin/
$./start-all.sh
注:hadoop里面的路径需要根据自己的实际路径做修改。