Hadoop配置
Docker
安装docker
$ yum install docker
设置开机自启Docker
$ systemctl enable docker
启动Docker
$ systemctl start docker
查看docker版本
[root@VM-16-13-centos ~]$ docker -v
"Docker version 1.13.1, build 0be3e21/1.13.1"
docker 配置阿里云镜像加速
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://i05bzcuy.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
java环境配置
首先在终端中输入 java 命令:
[root@localhost ~]$ java
-bash: java: command not found
使用wget工具获得JDK:(使用华为JDK镜像库):https://repo.huaweicloud.com/java/jdk/
如果没有weget,可以使用 yum install wget
$ wget https://repo.huaweicloud.com/java/jdk/12.0.1+12/jdk-12.0.1_linux-x64_bin.tar.gz
[root@localhost ~]# ll
total 185540
-rw-------. 1 root root 993 Jan 22 2018 anaconda-ks.cfg
-rw-r--r--. 1 root root 189981475 Apr 2 2019 jdk-12.0.1_linux-x64_bin.tar.gz
drwxr-xr-x. 2 root root 4096 Jun 6 2020 test
配置JAVA_HOME
$ cd /usr/local
$ mkdir java
$ mv jdk-12.0.1_linux-x64_bin.tar.gz /usr/local/java
$ cd java/
$ tar -zxvf jdk-12.0.1_linux-x64_bin.tar.gz
[root@localhost java]$ ll
total 185532
drwxr-xr-x. 9 root root 99 Dec 31 00:45 jdk-12.0.1
-rw-r--r--. 1 root root 189981475 Apr 2 2019 jdk-12.0.1_linux-x64_bin.tar.gz
$ cd jdk-12.0.1
$ vim /etc/profile
在文件底部添加如下内容:
# config the java home
JAVA_HOME=/usr/local/java/jdk-12.0.1
CLASSPATH=$JAVA_HOME/lib/
PATH=$PATH:$JAVA_HOME/bin
export PATH JAVA_HOME CLASSPATH
让配置生效:
source /etc/profile
检查安装:
$ java -version
java version "12.0.1" 2019-04-16
Java(TM) SE Runtime Environment (build 12.0.1+12)
Java HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)
docker 拉取centos内核镜像
$ docker search centos
$ docker pull centos
在拉取了centos镜像后,使用 docker images查看镜像,会有一个docker centos镜像在本地。
接下来,我们以该内核镜像来构建具有hadoop的容器。
docker 运行容器,配置hadoop,构建镜像
首先,我们在宿主机中应该准备好两个安装包
jdk-15.0.1_linux-x64_bin.tar.gz——JDK
hadoop-3.3.0.tar.gz——hadoop安装包
在当前目录中编辑 dockerfile文件dockerfile
FROM centos
MAINTAINER lengdanran
RUN yum install -y openssh-server sudo
RUN sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
RUN yum install -y openssh-clients
RUN echo "root:root" | chpasswd # 镜像容器的root用户密码为root,可以自己修改
RUN echo "root ALL=(ALL) ALL" >> /etc/sudoers # 授予root 用户容器的全部权限
RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key # ssh免密登录
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
RUN mkdir /var/run/sshd
EXPOSE 22 # 放开22端口
ADD jdk-15.0.1_linux-x64_bin.tar.gz /usr/local/ # 添加JDK,需要自行修改配置
RUN mv /usr/local/jdk-15.0.1 /usr/local/jdk
ENV JAVA_HOME /usr/local/jdk
ENV PATH $JAVA_HOME/bin:$PATH # 配置java 环境变量
ADD hadoop-3.3.0.tar.gz /usr/local # 添加 Hadoop
RUN mv /usr/local/hadoop-3.3.0 /usr/local/hadoop
ENV HADOOP_HOME /us