Docker的安装及Hadoop分布式的部署

最新推荐文章于 2024-05-03 22:29:39 发布
最新推荐文章于 2024-05-03 22:29:39 发布
阅读量600 收藏
点赞数 1
分类专栏： Hadoop 文章标签： Docker安装 Docker安装Hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/Victory_Lei/article/details/100926978
版权
大数据 同时被 3 个专栏收录
87 篇文章 0 订阅
订阅专栏
Hadoop
14 篇文章 0 订阅
订阅专栏
Docker
7 篇文章 0 订阅
订阅专栏

使用docker搭建部署hadoop分布式集群
在网上找了很长时间都没有找到使用docker搭建hadoop分布式集群的文档，没办法，只能自己写一个了。

一：环境准备：

1：首先要有一个Centos7操作系统，可以在虚拟机中安装。
a、root账户登录，查看内核版本
[root@admin01 ~]# uname -a
Linux admin01 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
b、更新yum源
[root@admin01 ~]# yum update
c、安装一些必要的系统工具
[root@admin01 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
d、添加软件源信息：
[root@admin01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
e、 更新 yum 缓存：
[root@admin01 ~]#  yum makecache fast
f、 可以查看所有仓库中所有docker版本，并选择特定版本安装
[root@admin01 ~]# yum list docker-ce --showduplicates | sort -r
g、安装Docker，命令：yum install docker-ce-版本号，我选的是17.12.1.ce
[root@admin01 ~]# yum install docker-ce-17.12.1.ce
h、验证安装是否成功(有client和service两部分表示docker安装启动都成功了)
[root@admin01 ~]# docker version
Client:
 Version:	17.12.0-ce
 API version:	1.35
 Go version:	go1.9.2
 Git commit:	c97c6d6
 Built:	Wed Dec 27 20:10:14 2017
 OS/Arch:	linux/amd64

Server:
 Engine:
  Version:	17.12.0-ce
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.9.2
  Git commit:	c97c6d6
  Built:	Wed Dec 27 20:12:46 2017
  OS/Arch:	linux/amd64
  Experimental:	false
i、启动Docker，命令：systemctl start docker，然后加入开机启动
[root@admin01 ~]# systemctl start docker
[root@admin01 ~]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
2：需要先构建一个hadoop的基础镜像，使用dockerfile文件方式进行构建。

先构建一个具备ssh功能的镜像，方便后期使用。（但是这样对于容器的安全性会有影响）
注意：这个镜像中的root用户的密码是root

[root@admin01 centos-ssh-root]# mkdir centos-ssh-root
[root@admin01 centos-ssh-root]# cd centos-ssh-root
[root@admin01 centos-ssh-root]# vi Dockerfile

其中Dockerfile里面的内容为：

[root@admin01 centos-ssh-root]# vi Dockerfile
    #添加
 # 选择一个已有的os镜像作为基础  
FROM centos 
   
# 镜像的作者  
MAINTAINER crxy 
   
# 安装openssh-server和sudo软件包，并且将sshd的UsePAM参数设置成no  
RUN yum install -y openssh-server sudo  
RUN sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config  
#安装openssh-clients
RUN yum  install -y openssh-clients
   
# 添加测试用户root，密码root，并且将此用户添加到sudoers里  
RUN echo "root:root" | chpasswd  
RUN echo "root   ALL=(ALL)       ALL" >> /etc/sudoers  
# 下面这两句比较特殊，在centos6上必须要有，否则创建出来的容器sshd不能登录  
RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key  
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key  
   
# 启动sshd服务并且暴露22端口  
RUN mkdir /var/run/sshd  
EXPOSE 22  
CMD ["/usr/sbin/sshd", "-D"]

构建命令：

[root@admin01 centos-ssh-root]# docker build -t="admin/centos-ssh-root" .
Sending build context to Docker daemon   2.56kB
Step 1/12 : FROM centos
 ---> 67fa590cfc1c
Step 2/12 : MAINTAINER crxy
 ---> Running in 6a6f50fca5b5
Removing intermediate container 6a6f50fca5b5
 ---> 5ab81450a24c
Step 3/12 : RUN yum install -y openssh-server sudo
 ---> Running in 7e2893515277
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
 * base: mirror.lzu.edu.cn
 * extras: mirrors.huaweicloud.com
 * updates: mirrors.cn99.com
Resolving Dependencies
--> Running transaction check
---> Package openssh-server.x86_64 0:7.4p1-16.el7 will be installed
--> Processing Dependency: openssh = 7.4p1-16.el7 for package: openssh-server-7.4p1-16.el7.x86_64
--> Processing Dependency: fipscheck-lib(x86-64) >= 1.3.0 for package: openssh-server-7.4p1-16.el7.x86_64
--> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-7.4p1-16.el7.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-server-7.4p1-16.el7.x86_64
---> Package sudo.x86_64 0:1.8.23-3.el7 will be installed
--> Running transaction check
---> Package fipscheck-lib.x86_64 0:1.4.1-6.el7 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.4.1-6.el7.x86_64
---> Package openssh.x86_64 0:7.4p1-16.el7 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package fipscheck.x86_64 0:1.4.1-6.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                  Arch          Version               Repository   Size
================================================================================
Installing:
 openssh-server           x86_64        7.4p1-16.el7          base        458 k
 sudo                     x86_64        1.8.23-3.el7          base        841 k
Installing for dependencies:
 fipscheck                x86_64        1.4.1-6.el7           base         21 k
 fipscheck-lib            x86_64        1.4.1-6.el7           base         11 k
 openssh                  x86_64        7.4p1-16.el7          base        510 k
 tcp_wrappers-libs        x86_64        7.6-77.el7            base         66 k

Transaction Summary
================================================================================
Install  2 Packages (+4 Dependent packages)

Total download size: 1.9 M
Installed size: 6.1 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/base/packages/fipscheck-1.4.1-6.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for fipscheck-1.4.1-6.el7.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total                                              1.4 MB/s | 1.9 MB  00:01     
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
 Userid     : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
 Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
 Package    : centos-release-7-6.1810.2.el7.centos.x86_64 (@CentOS)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : fipscheck-1.4.1-6.el7.x86_64                                 1/6 
  Installing : fipscheck-lib-1.4.1-6.el7.x86_64                             2/6 
  Installing : openssh-7.4p1-16.el7.x86_64                                  3/6 
  Installing : tcp_wrappers-libs-7.6-77.el7.x86_64                          4/6 
  Installing : openssh-server-7.4p1-16.el7.x86_64                           5/6 
  Installing : sudo-1.8.23-3.el7.x86_64                                     6/6 
  Verifying  : fipscheck-lib-1.4.1-6.el7.x86_64                             1/6 
  Verifying  : sudo-1.8.23-3.el7.x86_64                                     2/6 
  Verifying  : openssh-7.4p1-16.el7.x86_64                                  3/6 
  Verifying  : fipscheck-1.4.1-6.el7.x86_64                                 4/6 
  Verifying  : tcp_wrappers-libs-7.6-77.el7.x86_64                          5/6 
  Verifying  : openssh-server-7.4p1-16.el7.x86_64                           6/6 

Installed:
  openssh-server.x86_64 0:7.4p1-16.el7        sudo.x86_64 0:1.8.23-3.el7       

Dependency Installed:
  fipscheck.x86_64 0:1.4.1-6.el7      fipscheck-lib.x86_64 0:1.4.1-6.el7        
  openssh.x86_64 0:7.4p1-16.el7       tcp_wrappers-libs.x86_64 0:7.6-77.el7     

Complete!
Removing intermediate container 7e2893515277
 ---> a23041fa059e
Step 4/12 : RUN sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
 ---> Running in d5b7eb666c82
Removing intermediate container d5b7eb666c82
 ---> 8171ca2f7545
Step 5/12 : RUN yum  install -y openssh-clients
 ---> Running in 8d49b5c61e8b
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: mirror.lzu.edu.cn
 * extras: mirrors.huaweicloud.com
 * updates: mirrors.cn99.com
Resolving Dependencies
--> Running transaction check
---> Package openssh-clients.x86_64 0:7.4p1-16.el7 will be installed
--> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients-7.4p1-16.el7.x86_64
--> Running transaction check
---> Package libedit.x86_64 0:3.0-12.20121213cvs.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package              Arch        Version                       Repository
                                                                           Size
================================================================================
Installing:
 openssh-clients      x86_64      7.4p1-16.el7                  base      655 k
Installing for dependencies:
 libedit              x86_64      3.0-12.20121213cvs.el7        base       92 k

Transaction Summary
================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 747 k
Installed size: 2.8 M
Downloading packages:
--------------------------------------------------------------------------------
Total                                              940 kB/s | 747 kB  00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libedit-3.0-12.20121213cvs.el7.x86_64                        1/2 
  Installing : openssh-clients-7.4p1-16.el7.x86_64                          2/2 
  Verifying  : openssh-clients-7.4p1-16.el7.x86_64                          1/2 
  Verifying  : libedit-3.0-12.20121213cvs.el7.x86_64                        2/2 

Installed:
  openssh-clients.x86_64 0:7.4p1-16.el7                                         

Dependency Installed:
  libedit.x86_64 0:3.0-12.20121213cvs.el7                                       

Complete!
Removing intermediate container 8d49b5c61e8b
 ---> 6d7599dc2e56
Step 6/12 : RUN echo "root:root" | chpasswd
 ---> Running in 466cd81a8a2f
Removing intermediate container 466cd81a8a2f
 ---> 1d84010bd0d0
Step 7/12 : RUN echo "root   ALL=(ALL)       ALL" >> /etc/sudoers
 ---> Running in 238de2007a1e
Removing intermediate container 238de2007a1e
 ---> bdd3f6da541e
Step 8/12 : RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
 ---> Running in 369328c8512b
Enter passphrase (empty for no passphrase): Enter same passphrase again: Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
SHA256:aba5zQjMo9orNV5Jym8HyyQ+3pgWb74oy04bmEz5MuE root@369328c8512b
The key's randomart image is:
+---[DSA 1024]----+
|                 |
|                 |
|                 |
|  .   .  .       |
| + . o .S        |
|+oo Bo=o o       |
|oEo= X=oo        |
| o=+=*Bo.=       |
| .*BO*+oo o      |
+----[SHA256]-----+
Removing intermediate container 369328c8512b
 ---> 41ee2c8915b2
Step 9/12 : RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
 ---> Running in d820dfa82fb6
Enter passphrase (empty for no passphrase): Enter same passphrase again: Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
SHA256:qF+kJk9RQ0htp+MqzqhB/0+7z1S/LKaHoELbON2xjAk root@d820dfa82fb6
The key's randomart image is:
+---[RSA 2048]----+
|     ..o.        |
|      ..o .      |
|       .oo       |
|       oo.       |
| .    o.S. .     |
|. .E . *. . .    |
|. ..O Xo=..  .   |
| . Bo%o=+. +. .  |
|....=o++o++ .o   |
+----[SHA256]-----+
Removing intermediate container d820dfa82fb6
 ---> 4a3db8fa011f
Step 10/12 : RUN mkdir /var/run/sshd
 ---> Running in a8fd545374b4
Removing intermediate container a8fd545374b4
 ---> fe17d8a2889a
Step 11/12 : EXPOSE 22
 ---> Running in db2ad3e4383f
Removing intermediate container db2ad3e4383f
 ---> c3714cf390af
Step 12/12 : CMD ["/usr/sbin/sshd", "-D"]
 ---> Running in fbb7e94e8ef4
Removing intermediate container fbb7e94e8ef4
 ---> f957172bef4b
Successfully built f957172bef4b
Successfully tagged admin/centos-ssh-root:latest

在这里插入图片描述
查询刚才构建成功的镜像
在这里插入图片描述

3：基于这个镜像再构建一个带有jdk的镜像

注意：jdk使用的是1.8版本的

[root@admin01 ~]# mkdir centos-ssh-root-jdk
[root@admin01 ~]# cd centos-ssh-root-jdk/
[root@admin01 centos-ssh-root-jdk]# cp ../jdk-8u162-linux-x64.tar.gz .

其中Dockerfile里面的内容为：

[root@admin01 centos-ssh-root-jdk]# vi Dockerfile
 ##添加
FROM admin/centos-ssh-root
ADD jdk-8u162-linux-x64.tar.gz /usr/local/
RUN mv /usr/local/jdk1.8.0_162 /usr/local/jdk1.8
ENV JAVA_HOME /usr/local/jdk1.8
ENV PATH $JAVA_HOME/bin:$PATH

构建命令：

[root@admin01 centos-ssh-root-jdk]# docker build -t="admin/centos-ssh-root-jdk" .
Sending build context to Docker daemon  189.8MB
Step 1/5 : FROM admin/centos-ssh-root
 ---> f957172bef4b
Step 2/5 : ADD jdk-8u162-linux-x64.tar.gz /usr/local/
 ---> 4c6b1cb6fea0
Step 3/5 : RUN mv /usr/local/jdk1.8.0_162 /usr/local/jdk1.8
 ---> Running in c9cde9060bc4
Removing intermediate container c9cde9060bc4
 ---> 83a08f6c8444
Step 4/5 : ENV JAVA_HOME /usr/local/jdk1.8
 ---> Running in faabe30cc8af
Removing intermediate container faabe30cc8af
 ---> 8ce4aeda5c6e
Step 5/5 : ENV PATH $JAVA_HOME/bin:$PATH
 ---> Running in a6695cfd7862
Removing intermediate container a6695cfd7862
 ---> 39bab29b31af
Successfully built 39bab29b31af
Successfully tagged admin/centos-ssh-root-jdk:latest

在这里插入图片描述
查询构建成功的镜像
在这里插入图片描述

4：基于这个jdk镜像再构建一个带有hadoop的镜像

注意：hadoop使用的是2.7.3版本的。

[root@admin01 ~]# 
[root@admin01 ~]# mkdir centos-ssh-root-jdk-hadoop
[root@admin01 ~]# cd centos-ssh-root-jdk-hadoop/
[root@admin01 centos-ssh-root-jdk-hadoop]# cp ../hadoop-2.7.3.tar.gz .
[root@admin01 centos-ssh-root-jdk-hadoop]# vi Dockerfile

其中，Dockerfile里面的内容为：

[root@admin01 centos-ssh-root-jdk-hadoop]# vi Dockerfile
## 添加
FROM admin/centos-ssh-root-jdk
ADD hadoop-2.7.3.tar.gz /usr/local
RUN mv /usr/local/hadoop-2.7.3 /usr/local/hadoop
ENV HADOOP_HOME /usr/local/hadoop
ENV PATH $HADOOP_HOME/bin:$PATH

构建命令：

[root@admin01 centos-ssh-root-jdk-hadoop]# docker build -t="admin/centos-ssh-root-jdk-hadoop" .
Sending build context to Docker daemon  214.1MB
Step 1/5 : FROM admin/centos-ssh-root-jdk
 ---> 39bab29b31af
Step 2/5 : ADD hadoop-2.7.3.tar.gz /usr/local
 ---> a565f87323c1
Step 3/5 : RUN mv /usr/local/hadoop-2.7.3 /usr/local/hadoop
 ---> Running in 5f07e5a1d490
Removing intermediate container 5f07e5a1d490
 ---> ff2197105cde
Step 4/5 : ENV HADOOP_HOME /usr/local/hadoop
 ---> Running in 11d8a893b137
Removing intermediate container 11d8a893b137
 ---> 46061a36967e
Step 5/5 : ENV PATH $HADOOP_HOME/bin:$PATH
 ---> Running in acfd517cb020
Removing intermediate container acfd517cb020
 ---> bc32364422f6
Successfully built bc32364422f6
Successfully tagged admin/centos-ssh-root-jdk-hadoop:latest

在这里插入图片描述
查询构建成功的镜像
在这里插入图片描述

二、搭建hadoop分布式集群

1：集群规划
准备搭建一个具有三个节点的集群，一主两从
主节点：admin01 ip：192.168.47.101
从节点1：admin02 ip：192.168.47.102
从节点2：admin03 ip：192.168.47.103

但是由于docker容器重新启动之后ip会发生变化，所以需要我们给docker设置固定ip。使用pipework给docker容器设置固定ip
2：启动三个容器，分别作为admin01、admin02、admin03
在宿主机上执行下面命令，给容器设置主机名和容器的名称，并且在admin01中对外开放端口50070 和8088

[root@admin01 ~]#   docker run --name admin01 --hostname admin01 -d -P -p 50070:50070 -p 8088:8088 admin/centos-ssh-root-jdk-hadoop
  
[root@admin01 ~]# docker run --name admin02 --hostname admin02 -d -P admin/centos-ssh-root-jdk-hadoop

[root@admin01 ~]# docker run --name admin03 --hostname admin03 -d -P admin/centos-ssh-root-jdk-hadoop

查看运行的实例：

[root@admin01 ~]# docker ps
CONTAINER ID        IMAGE                              COMMAND               CREATED              STATUS              PORTS                                                                     NAMES
4083a6ab0808        admin/centos-ssh-root-jdk-hadoop   "/usr/sbin/sshd -D"   49 seconds ago       Up 49 seconds       0.0.0.0:32770->22/tcp                                                     admin03
6e2e2f0a40fb        admin/centos-ssh-root-jdk-hadoop   "/usr/sbin/sshd -D"   About a minute ago   Up About a minute   0.0.0.0:32769->22/tcp                                                     admin02
d4f382e8579c        admin/centos-ssh-root-jdk-hadoop   "/usr/sbin/sshd -D"   2 hours ago          Up 2 hours          0.0.0.0:8088->8088/tcp, 0.0.0.0:50070->50070/tcp, 0.0.0.0:32768->22/tcp   admin01

在这里插入图片描述

3：给这三台容器设置固定IP
1：下载pipework

下载地址：https://github.com/jpetazzo/pipework.git

2：把下载的zip包上传到宿主机服务器上，解压,并且修改名字
[root@admin01 ~]# unzip pipework-master.zip 
Archive:  pipework-master.zip
ae42f1b5fef82b3bc23fe93c95c345e7af65fef3
   creating: pipework-master/
 extracting: pipework-master/.gitignore  
  inflating: pipework-master/LICENSE  
  inflating: pipework-master/README.md  
  inflating: pipework-master/docker-compose.yml  
   creating: pipework-master/doctoc/
  inflating: pipework-master/doctoc/Dockerfile  
  inflating: pipework-master/pipework  
  inflating: pipework-master/pipework.spec  
[root@admin01 ~]# mv pipework-master pipework
[root@admin01 ~]# cp -rp pipework/pipework /usr/local/bin/
3：安装bridge-utils
[root@admin01 ~]# yum -y install bridge-utils
4:创建网络
[root@admin01 ~]# brctl addbr br0
[root@admin01 ~]# ip link set dev br0 up
[root@admin01 ~]# ip addr add 192.168.47.101/24 dev br0
5：给容器设置固定ip
[root@admin01 ~]# pipework br0 admin01 192.168.47.101/24
[root@admin01 ~]# pipework br0 admin02 192.168.47.102/24
[root@admin01 ~]# pipework br0 admin03 192.168.47.103/24

验证一下，分别ping三个ip，能ping通就说明没问题
在这里插入图片描述

4: 配置Hadoop集群

先连接到admin01上，使用命令

[root@admin01 ~]# docker exec -it admin01 /bin/bash

下面的步骤就是hadoop集群的配置过程

1：设置主机名与ip的映射，修改三台容器：vi /etc/hosts

添加下面配置

192.168.47.101	admin01
192.168.47.102	admin02
192.168.47.103	admin03
2：设置ssh免密码登录

在admin01上执行下面操作

Rocky000000
关注
专栏目录
Hadoop Docker 安装
weixin_45526462的博客
07-14 873
准备JDK安装包下载Hadoop，官网提供Source和Binary两种方式，这里采用的是Binary的Hadoop-3.3.3.tar.gz。停服问题准备DockerFile 等待更新… 4 5 6 7 8 9 9docker run -itd --network 网络名 --ip 地址 -p 端口:22 -p 端口:9870 -p 端口:9000 -p 端口:9864 -p 端口:9866 -p 端口:8088 -v /usr/local/docker-data/地址:/usr/local/docker
Docker + Hadoop 搭建完全分布式
2401_84413092的博客
04-28 607
开发工具安装。
docker安装hadoop
weixin_33738982的博客
05-05 110
　　 　　docker为hadoop的云化带来了极大便利,安装和应用也会更快更方便.进入正题: docker search hadoop 　　将会看到如下结果: INDEX NAME DESCRIPTION STARS ...
Docker 安装hadoop
中国红客99代传人的博客
05-10 516
硬件：单核CPU、内存1G 软件：Ubuntu 16.04操作系统、Docker、Hadoop。
docker 安装hadoop
路在脚下
07-24 1197
docker 安装hadoop，通过docker自带hadoop方便适用
使用 Docker 搭建 Hadoop 分布式环境_windows系统docker怎么搭建hadoop框架
最新发布
2401_84182318的博客
05-03 546
下载这个升级包，并安装之后点击 Docker desktop 安装报错提示框中的Restart，就能顺利安装完成并启动了。
Docker Compose部署Hadoop分布式环境与MapReduce应用
本资源主要聚焦于Hadoop的分布式编程和容器化部署...这个资源涵盖了如何使用Docker和docker-compose来简化Hadoop分布式编程的部署过程，强调了容器化带来的便利性和管理优势，这对于理解和实践Hadoop集群运维至关重要。
使用Docker快速搭建Hadoop分布式集群实战指南
"使用Docker搭建Hadoop分布式集群...利用Docker搭建Hadoop分布式集群可以极大地简化部署流程，提高效率，并允许快速实验不同的配置。只要遵循上述步骤，即使是对Hadoop不熟悉的用户也能轻松建立自己的测试或生产环境。
docker一键安装hadoop集群
10-14
全部源码dockerfile等，按照readme说明可以使用docker一键安装hadoop集群，需要在linux系统下使用
Docker安装Hadoop
m0_67402013的博客
08-14 3759
docker exec -it hadoop bash进入容器。查找hadoop-env.sh的安装路径。配置备份数量，小于等于slave数量。三个容器都要修改如下三个IP映射。
docker安装Hadoop
kongshuo的博客
02-01 629
docker安装Hadoop
使用 Docker 安装 Hadoop 集群
m0_67400973的博客
08-27 2273
相信大家都有过玩虚拟机的经验，也相信大家有想搭一个大数据集群偶尔在自己电脑上做做测试。使用虚拟机是可以完成的，但是过程非常漫长，而且很多操作不能以后重复使用。这里我近期使用Docker完成了测试集群的搭建，我想用的时候，十几秒就可以直接启动，速度很快。而且所有的镜像都可以上传hub云端，想用的时候，从云端pull到任何一台有docker的电脑直接运行。而且，以往我们使用虚拟机搭建集群，我们需要配置很多的免密，甚至Hadoop等文件，我们需要上传到各个节点上。...
在docker中安装Hadoop
王茗灏的专栏
07-04 781
 tar -xzvf zookeeper-3.4.11.tar.gz/opt/zookeeper-3.4.12配置环境变量vim /etc/profilve        # 添加如下内容        export ZOOKEEPER_HOME = /opt/zookeeper-3.4.12        export PATH = $PATH:$ZOOKEEPER_HOME/bin配置zook...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值