Hadoop伪分布集群环境的搭建

最新推荐文章于 2022-09-13 16:20:07 发布
最新推荐文章于 2022-09-13 16:20:07 发布
阅读量250 收藏 2
点赞数 2
分类专栏： 笔记
版权声明：本文为博主原创文章，遵循 CC 4.0 BY 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/wkt1105436760/article/details/115312921
版权

Hadoop伪分布集群环境的搭建

一、关闭防火墙和禁用SELINUX

1、关闭防火墙
[root@bogon ~]# systemctl stop firewalld.service

[root@bogon ~]# systemctl status firewalld.service
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)

3月 30 05:58:13 localhost.localdomain systemd[1]: Starting firewalld - dynam...
3月 30 05:58:14 localhost.localdomain systemd[1]: Started firewalld - dynami...
3月 30 05:58:15 localhost.localdomain firewalld[777]: WARNING: ICMP type 'be...
3月 30 05:58:15 localhost.localdomain firewalld[777]: WARNING: beyond-scope:...
3月 30 05:58:15 localhost.localdomain firewalld[777]: WARNING: ICMP type 'fa...
3月 30 05:58:15 localhost.localdomain firewalld[777]: WARNING: failed-policy...
3月 30 05:58:15 localhost.localdomain firewalld[777]: WARNING: ICMP type 're...
3月 30 05:58:15 localhost.localdomain firewalld[777]: WARNING: reject-route:...
3月 29 22:03:42 bogon systemd[1]: Stopping firewalld - dynamic firewall da.....
3月 29 22:03:42 bogon systemd[1]: Stopped firewalld - dynamic firewall daemon.
Hint: Some lines were ellipsized, use -l to show in full.
2、禁用SELINUX

运行命令sudo vim /etc/sysconfig/selinux
找到行SELINUX=enforcing
替换为SELINUX=disabled
保存退出
reboot


# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

三、 配置hostname与IP地址之间的对应关系

[root@localhost ~]# vi /etc/hosts
[root@localhost ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.110 localhost

四、配置SSH免密码登录

[hadoop@localhost ~]$ mkdir .ssh
[hadoop@localhost ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:gs36CppDtcXbrBFQme8H8pYZ523rnZi9287QDCj8Zyc hadoop@localhost.localdomain
The key's randomart image is:
+---[RSA 2048]----+
|    ..o          |
|   . o           |
|    o .          |
|   . O +..  .    |
|  . + % So.. .   |
| . . + X ooo  +  |
|. . . + . ...E + |
|.o . o     .B B  |
|o.  ...   .+ *++ |
+----[SHA256]-----+
[hadoop@localhost ~]$ cd .ssh/
[hadoop@localhost .ssh]$ ls
id_rsa  id_rsa.pub
[hadoop@localhost .ssh]$ cat id_rsa.pub >> authorized_keys
[hadoop@localhost .ssh]$ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCigTeuw1spXoaqwdsROjamKwSsbZmtB1QZe2cvM6PVYu+bTTt47f7VyGQWZJqUz45QsJ5yAUwATHDHbDk0XVdXr8S6YDI/ON9aGDHNS316UNU2+gBwd52k1fnf/HKu7ivSuA1IKQwWkYudkSyUyg1yb+WHb7PlMsBXl+r909I9NrhOP+MT1SNeQMkiuDFjghpJg0abymECblwxIdZtG50cHE4xVlk8W2ZHECiAfvl6EjcYR0+SG2fi5QIVlCw7bJGkdan8maXSfLD3aGWeRKhKcebBdmQtF+4MJsiq0of+CAznfqvoTeklWL3gifmk1Akm7HDD8UlrDJ9ed4DDnvsv hadoop@localhost.localdomain
[hadoop@localhost .ssh]$ cd ..
[hadoop@localhost ~]$ chmod 700 .ssh
[hadoop@localhost ~]$ chmod 600 .ssh/*
[hadoop@localhost ~]$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:8tfa8326wjdlr8z6PLJh70gZ+RUlr3+QAN4g3qdp66o.
ECDSA key fingerprint is MD5:e3:25:85:67:f1:4c:45:04:6d:03:4f:41:f2:43:a3:73.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Last login: Mon Mar 29 22:23:11 2021
[hadoop@localhost ~]$ exit
登出
Connection to localhost closed.
[hadoop@localhost ~]$ ssh localhost
Last login: Mon Mar 29 22:26:17 2021 from ::1

五、JDK的安装

[hadoop@localhost ~]$ ls
hadoop-2.7.3.tar.gz              公共  视频  文档  音乐
jdk-14.0.2_linux-x64_bin.tar.gz  模板  图片  下载  桌面
[hadoop@localhost ~]$ mkdir app
[hadoop@localhost ~]$ cp jdk-14.0.2_linux-x64_bin.tar.gz app
[hadoop@localhost ~]$ cd app
[hadoop@localhost app]$ ls
jdk-14.0.2_linux-x64_bin.tar.gz
[hadoop@localhost app]$ tar -zxvf jdk-14.0.2_linux-x64_bin.tar.gz 
jdk-14.0.2/bin/jaotc
jdk-14.0.2/bin/jar
jdk-14.0.2/bin/jarsigner
jdk-14.0.2/bin/java
jdk-14.0.2/bin/javac
jdk-14.0.2/bin/javadoc
jdk-14.0.2/bin/javap
jdk-14.0.2/bin/jcmd
jdk-14.0.2/bin/jconsole
jdk-14.0.2/bin/jdb
jdk-14.0.2/bin/jdeprscan
......
[hadoop@localhost app]$ ls
jdk-14.0.2  jdk-14.0.2_linux-x64_bin.tar.gz
[hadoop@localhost app]$ ln -s jdk-14.0.2 jdk
[hadoop@localhost app]$ ls
jdk  jdk-14.0.2  jdk-14.0.2_linux-x64_bin.tar.gz
[hadoop@localhost app]$ vi ~/.bashrc
[hadoop@localhost app]$ cat ~/.bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions
JAVA_HOME=/home/hadoop/app/jdk
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME CLASSPATH PASH 
[hadoop@localhost app]$ source ~/.bashrc
[hadoop@localhost app]$ java -version
java version "14.0.2" 2020-07-14
Java(TM) SE Runtime Environment (build 14.0.2+12-46)
Java HotSpot(TM) 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing)

六、Hadoop伪分布集群的安装配置

1、解压进入hadoop配置文件目录
[hadoop@localhost ~]$ cp hadoop-2.7.3.tar.gz app
[hadoop@localhost ~]$ cd app
[hadoop@localhost ~]$ tar -zxvf hadoop-2.7.3.tar.gz
.......
[hadoop@localhost app]$ cd
[hadoop@localhost ~]$ cd app/hadoop-2.7.3/etc/hadoop
[hadoop@localhost hadoop]$ ls
capacity-scheduler.xml      httpfs-env.sh            mapred-env.sh
configuration.xsl           httpfs-log4j.properties  mapred-queues.xml.template
container-executor.cfg      httpfs-signature.secret  mapred-site.xml.template
core-site.xml               httpfs-site.xml          slaves
hadoop-env.cmd              kms-acls.xml             ssl-client.xml.example
hadoop-env.sh               kms-env.sh               ssl-server.xml.example
hadoop-metrics2.properties  kms-log4j.properties     yarn-env.cmd
hadoop-metrics.properties   kms-site.xml             yarn-env.sh
hadoop-policy.xml           log4j.properties         yarn-site.xml
hdfs-site.xml               mapred-env.cmd

2、修改core-site.xml，hdfs-site.xml,hadoop-env.sh,mapred-site.xml,yarn-site.xml,slaves配置文件

hadoop-env.sh中只改这一条
export JAVA_HOME=/home/hadoop/app/jdk

[hadoop@localhost hadoop]$ vi core-site.xml
[hadoop@localhost hadoop]$ cat core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
               <name>fs.defaultFS</name>
               <value>hdfs://localhost:9000</value>
</property>
<property>
               <name>hadoop.tmp.dir</name>
               <value>file:/home/hadoop/data/tmp</value>
</property>
<property>
               <name>hadoop.proxyuser.hadoop.hosts</name>
               <value>*</value>
</property>
<property>
               <name>hadoop.proxyuser.hadoop.groups</name>
               <value>*</value>
</property>
</configuration>


[hadoop@localhost hadoop]$ vi hdfs-site.xml
[hadoop@localhost hadoop]$ cat hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
         <name>dfs.namenode.name.dir</name>
         <value>/home/hadoop/data/dfs/name</value>   
         <final>true</final>
</property>
<property>
         <name>dfs.datanode.data.dir</name>  
         <value>/home/hadoop/data/dfs/data</value>
         <final>true</final>
</property>
<property>
         <name>dfs.replication</name>
         <value>1</value>   
</property>
<property>
         <name>dfs.permissions</name> 
         <value>false</value>
</property>

</configuration>



[hadoop@localhost hadoop]$ vi hadoop-env.sh
[hadoop@localhost hadoop]$ cat hadoop-env.sh
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Set Hadoop-specific environment variables here.

# The only required environment variable is JAVA_HOME.  All others are
# optional.  When running a distributed configuration it is best to
# set JAVA_HOME in this file, so that it is correctly defined on
# remote nodes.

# The java implementation to use.
export JAVA_HOME=/home/hadoop/app/jdk

# The jsvc implementation to use. Jsvc is required to run secure datanodes
# that bind to privileged ports to provide authentication of data transfer
# protocol.  Jsvc is not required if SASL is configured for authentication of
# data transfer protocol using non-privileged ports.
#export JSVC_HOME=${JSVC_HOME}

export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}

# Extra Java CLASSPATH elements.  Automatically insert capacity-scheduler.
for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do
  if [ "$HADOOP_CLASSPATH" ]; then
    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f
  else
    export HADOOP_CLASSPATH=$f
  fi
done

# The maximum amount of heap to use, in MB. Default is 1000.
#export HADOOP_HEAPSIZE=
#export HADOOP_NAMENODE_INIT_HEAPSIZE=""

# Extra Java runtime options.  Empty by default.
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"

# Command specific options appended to HADOOP_OPTS when specified
export HADOOP_NAMENODE_OPTS="-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_NAMENODE_OPTS"
export HADOOP_DATANODE_OPTS="-Dhadoop.security.logger=ERROR,RFAS $HADOOP_DATANODE_OPTS"

export HADOOP_SECONDARYNAMENODE_OPTS="-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_SECONDARYNAMENODE_OPTS"

export HADOOP_NFS3_OPTS="$HADOOP_NFS3_OPTS"
export HADOOP_PORTMAP_OPTS="-Xmx512m $HADOOP_PORTMAP_OPTS"

# The following applies to multiple commands (fs, dfs, fsck, distcp etc)
export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
#HADOOP_JAVA_PLATFORM_OPTS="-XX:-UsePerfData $HADOOP_JAVA_PLATFORM_OPTS"

# On secure datanodes, user to run the datanode as after dropping privileges.
# This **MUST** be uncommented to enable secure HDFS if using privileged ports
# to provide authentication of data transfer protocol.  This **MUST NOT** be
# defined if SASL is configured for authentication of data transfer protocol
# using non-privileged ports.
export HADOOP_SECURE_DN_USER=${HADOOP_SECURE_DN_USER}
export HADOOP_IDENT_STRING=$USER



[hadoop@localhost hadoop]$ vi mapred-site.xml
[hadoop@localhost hadoop]$ cat mapred-site.xml
<configuration>
		<property>
			<name>mapreduce.framework.name</name>
			<value>yarn</value>
		</property>
</configuration>


[hadoop@localhost hadoop]$ vi yarn-site.xnl
[hadoop@localhost hadoop]$ cat yarn-site.xnl
censed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>
	<property>
			<name>yarn.nodemanager.aux-services</name>  
			<value>mapreduce_shuffle</value>  
		</property>
</configuration
[hadoop@localhost hadoop]$ vi yarn-site.xnl
[hadoop@localhost hadoop]$ cat yarn-site.xnl
censed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>
	<property>
			<name>yarn.nodemanager.aux-services</name>  
			<value>mapreduce_shuffle</value>  
		</property>
</configuration>



[hadoop@localhost hadoop]$ cat slaves
localhost


[hadoop@localhost ~]$ cd app
[hadoop@localhost app]$ ls
hadoop-2.7.3         jdk         jdk-14.0.2_linux-x64_bin.tar.gz
hadoop-2.7.3.tar.gz  jdk-14.0.2
[hadoop@localhost app]$ ln -s hadoop-2.7.3 hadoop
[hadoop@localhost app]$ ls
hadoop        hadoop-2.7.3.tar.gz  jdk-14.0.2
hadoop-2.7.3  jdk                  jdk-14.0.2_linux-x64_bin.tar.gz
[hadoop@localhost app]$ vi ~/.bashrc
[hadoop@localhost app]$ cat ~/.bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions
JAVA_HOME=/home/hadoop/app/jdk
HADOOP_HOME=/home/hadoop/app/hadoop
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$JAVA_HOME/bin/:HADOOP_HOME/bin:$PATH
export JAVA_HOME CLASSPATH PASH HADOOP_HOME
[hadoop@localhost app]$ source ~/.bashrc
[hadoop@localhost app]$ cd
[hadoop@localhost ~]$ ls
app                  jdk-14.0.2_linux-x64_bin.tar.gz  模板  图片  下载  桌面
hadoop-2.7.3.tar.gz  公共                             视频  文档  音乐
[hadoop@localhost ~]$ mkdir -p data/tmp
[hadoop@localhost ~]$ mkdir -p data/dfs/name
[hadoop@localhost ~]$ mkdir -p data/dfs/data
[hadoop@localhost ~]$ ls
app   hadoop-2.7.3.tar.gz              公共  视频  文档  音乐
data  jdk-14.0.2_linux-x64_bin.tar.gz  模板  图片  下载  桌面
[hadoop@localhost ~]$ cd app/hadoop
[hadoop@localhost hadoop]$ cd bin
[hadoop@localhost bin]$ ./hadoop namenode -format
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

21/03/29 23:02:38 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = localhost.localdomain/127.0.0.1
STARTUP_MSG:   args = [-format]
......
[hadoop@localhost hadoop]$ jps
3172 DataNode
3609 NodeManager
3501 ResourceManager
3901 Jps
1312 NameNode
1567 SecondaryNode
木头科技
关注
  • 2
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 2
    评论
专栏目录
启动hadoop报异常Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
zcs2312852665的博客
11-24 934
执行以下命令，然后连续点三下回车生成SSH密匙对。原因分析：没有生成、配置公匙，分发给自己。把密匙对分发给本机，让本机可以免密登录。然后再去重新启动hadoop。
零基础也能学会！Hadoop伪分布式集群安装与配置实践
最新发布
weixin_64289779的博客
04-30 895
Hadoop是典型的分布式系统，原则上需要在至少3个节点构成集群上进行安装完全分布式系统。但是，出于学习的目的，在资源受限的情况下，可以有简化的安装模式，比如由单个节点构成的伪分布式安装(仍然通过网络进行节点访问，但是访问的IP地址：127.0.0.1就是本机地址)。
Hbase常见问题
奔跑的豆子的专栏
01-16 1641
常见问题 问题一 localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. 解决：执行SSH设置和密钥生成操作 问题二 Unable to load native-hadoop library for your platform... using builti
hadoop环境搭建之伪分布集群环境搭建（单节点）
Marlon1993的博客
03-02 246
首先，尼玛哥是一名研究生，可以说很多学习的知识都是来源于同门师兄弟，本着将自己学习到的东西以博客的形式分享到网络上，不仅仅可以让自己回顾，也可以帮助需要学习的小伙伴们，供大家一同学习。 首先，hadoop 环境的搭建需要在linux的环境下，那么，现在，咱们需要将linux的基本网络设置已经配置完毕，注意，每一次配置完成，建议要给系统做一个快照，以免不...
hadoop 3.x 伪集群安装
笔生花的博客
09-13 471
hadoop新版本默认以root用户启动会出错的，然而大多数时候我们安装的时候习惯用root，毕竟时自己的测试机器。vim /soft/hadoop/hadoop-3.2.3/etc/hadoop/hadoop-env.sh //请改成自己的解压位置。java 用的：jdk-8u321-linux-x64.tar.gz //请自行下载。进入目录：cd /soft/hadoop/hadoop-3.2.3/etc/hadoop。workers中内容清空，加入你的主机名，在这里我加的是master。
Hadoop伪分布式集群环境搭建
10-13
Hadoop伪分布式集群环境搭建，有常见错误及解决方案。
Hadoop伪分布模式环境搭建[超详细]
07-15
Hadoop伪分布模式环境搭建是指在单个节点上搭建Hadoop集群环境，主要用于开发调试MapReduce应用逻辑。以下是搭建伪分布模式环境的步骤： 一、SSH协议介绍 SSH（Secure Shell）是一种安全协议，用于远程登录服务器...
linux虚拟机搭建hadoop伪分布式集群
04-26
在搭建Hadoop伪分布式集群的过程中，首先需要在Linux环境下配置VMware的NAT网络，确保虚拟机能够稳定地运行和通信。NAT网络允许虚拟机通过宿主机与外部网络进行通信，同时保持IP地址的固定性。具体操作包括取消DHCP...
大数据+Hadoop3.X伪分布式集群搭建笔记+练习搭建Hadoop平台
08-08
练习搭建伪分布Hadoop3.X集群，只用于刚刚开始学习搭建hadoo伪分布式集群的人群，帮助大家快速搭建Hadoop3.X伪分布式集群，快速入门大数据为日后的学习打下坚实的基础
hadoop cdh版本伪分布式集群搭建图文教程
08-18
hadoop cdh版本伪分布式集群搭建详细图文教程
hadoop的单机伪分布安装笔记
04-28
hadoop的单机伪分布安装笔记,使用虚拟机方式、端口转发、搭建hadoop的单机环境。学习hadoop的入门文档之一。
hadoop集群搭建篇（伪分布，完全分布）
eyeofeagle的博客
08-01 2974
要点如下： 1，服务器搭建： virtualBox + Centos7 2, centos7: 网络配置 3，centos 上安装jdk, hadoop 第一步： 安装centos, 配置网络 我使用ubuntu16 桌面版， 安装virtualbox后，创建虚拟机： 添加centos7镜像文件 点击 ‘start’， 开始安装centos7.... 安装完成后， 需要关机，c...............
hadoop-3.2.1伪分布集群安装
huangbaokang的博客
05-28 778
准备环境： 一台虚拟机（centos7） ip:192.168.8.124 hostname:hadoop124 一、前提环境准备 1.设置静态ip 设置好后，需要重启网络 service network restart 2.设置主机名 临时设置主机名 永久设置主机名 3.hosts文件配置 4.关闭防火墙 注意，centos6跟centos7命令不同 临时关闭防火墙 systemctl stop firewalld 永久关闭防火墙 systemctl disable firewalld 5.
hadoop伪分布式集群搭建
雷恩Layne
03-21 1328
本文步骤较多，请细心查看。 基础设施 基础设施环境如下： jdk 1.7+（提前设置好环境变量） ssh自己和自己之间进行免密登陆，如在layne1上执行ssh layne1 时间同步 设置本机ip 设置主机名 可参考Linux切换运行级别、关闭防火墙、禁用selinux、关闭sshd、时间同步、修改时区、拍摄快照、克隆操作、修改语言环境。 另外，不得不提Linux系统远程执行和远程登陆的区别： 远程执行：不需要用户交互，而是用户直接给出一个命令，直接在远程执行，不会加载 /etc/profile.
hadoop伪分布搭建的步骤
一库大神の博客
04-10 393
1.1 设置ip地址 执行命令：service network restart 验证：ifconfig 1.2 关闭防火墙 执行命令：service iptables stop 验证：service iptables status 1.3关闭防火墙的自动运行 执行命令：chkconfig iptables off 验证：chkconfig --lis
Apache hadoop集群安装的三种方式：本地、伪分布、完全分布
qq_37873221的博客
05-08 3944
四 Hadoop运行模式1）官方网址（1）官方网站：http://hadoop.apache.org/（2）各个版本归档库地址 https://archive.apache.org/dist/hadoop/common/hadoop-2.7.2/（3）hadoop2.7.2版本详情介绍http://hadoop.apache.org/docs/r2.7.2/2）Hadoop运行模式（1）本地模式（...
Hadoop伪分布式集群搭建
qq_44405452的博客
05-04 2167
安装JDK （1）准备软件 准备jdk安装包： jdk-8u221-linux-x64.tar.gz （2）解压压缩包 tar -zxvf jdk-8u221-linux-x64.tar.gz （3）在此处我们配置系统环境变量，使用命令： vim /etc/profile （4）在最后加入以下两行内容： export JAVA_HOME=/root/software/jdk1.8.0_221 # 配置Java的安装目录，我的在/root/software文件夹下 export PATH=$PATH:$
Hadoop集群搭建（伪分布式）
weixin_56386353的博客
07-15 1927
集群搭建hadoop伪分布式 软件准备： VMware Workstation Pro centos 7 xshell5 jdk-8u141-linux-x64.tar.gz hadoop-2.10.1.tar.gz 一.安装jdk 1.首先在jdk官网中，下载jdk安装包，本文以jdk-8u141-linux-x64.tar.gz为例 2.将安装包传入虚拟机中，本文以xshell5为例 （1）.打开xshell5，连接上虚拟机，连接成功之后会有以下提示 （2）.输入rz命令，出现传输文件的提示框，若没有
自制Hadoop伪分布、集群安装详细过程（vmware）
Fortuna_i的博客
07-20 2239
Hadoop单机版、伪分布、集群安装教程推荐链接： http://dblab.xmu.edu.cn/blog/page/2/?s=Hadoop 注：vmware6.5及以上，执行命令基本相同，除了编辑文件时“vi”指令替换为“vim”指令 一、下载安装所需文件          1.vmware10中文版          2.centos6.4          3.jdk1.8-L...
Hadoop CDH伪分布集群搭建实战教程
本文档是一份详尽的Hadoop CDH版本伪分布式集群搭建教程，针对初学者设计，旨在帮助读者快速掌握大数据技术的基础环境构建。以下是主要知识点的详细说明： 1. **所需软件和资源**：教程开始首先提到，参与者需要...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

木头科技

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值