Windows PC上创建大数据职业技能竞赛实验环境之四--客户端环境的搭建

liu9ang

已于 2022-04-13 16:12:29 修改

阅读量2.2k

点赞数

分类专栏：大数据平台文章标签： hadoop intellij-idea

于 2022-04-13 11:37:35 首次发布

本文链接：https://blog.csdn.net/liu9ang/article/details/124090526

版权

本文介绍了在Windows PC上使用Ubuntu Docker容器搭建Hadoop客户端环境的步骤，包括安装SSH、JDK，配置JetBrains IntelliJ IDEA，拷贝Hadoop集群环境，并启用X11 forwarding。同时，还涉及了通过MobaXterm进行SSH连接和设置，以及安装ping等必备工具。

摘要由CSDN通过智能技术生成

Hadoop集群访问客户端
X window
Jet brain Idea和客户端开发环境

以上环境我们将在WSL的Ubuntu docker容器中实现，其中对X和Idea的访问，通过Windows Xserver实现。

1 拉取Ubuntu镜像，运行ubuntu容器。

wslu@LAPTOP-ERJ3P24M:~$ sudo docker pull ubuntu:latest
wslu@LAPTOP-ERJ3P24M:~$ sudo docker run -it --name hadoop-client -p 22222:22 -v /home/wslu/data:/root/data ubuntu:latest
 /bin/bash

修改ubuntu软件源为阿里云，将archive.ubuntu.com和security.ubuntu.com替换为mirrors.aliyun.com。

root@19bafff6decc:/# cd /etc/apt
root@19bafff6decc:/etc/apt# ls
apt.conf.d  auth.conf.d  preferences.d  sources.list  sources.list.d  trusted.gpg.d
root@19bafff6decc:/etc/apt# cp sources.list sources.list__
root@19bafff6decc:/etc/apt# vi sources.list

root@19bafff6decc:~# apt update

Get:1 http://mirrors.aliyun.com/ubuntu focal InRelease [265 kB]
Get:2 http://mirrors.aliyun.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://mirrors.aliyun.com/ubuntu focal-backports InRelease [108 kB]
Get:4 http://mirrors.aliyun.com/ubuntu focal-security InRelease [114 kB]
Get:5 http://mirrors.aliyun.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:6 http://mirrors.aliyun.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:7 http://mirrors.aliyun.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:8 http://mirrors.aliyun.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:9 http://mirrors.aliyun.com/ubuntu focal-updates/main amd64 Packages [2146 kB]
Get:10 http://mirrors.aliyun.com/ubuntu focal-updates/multiverse amd64 Packages [30.3 kB]
Get:11 http://mirrors.aliyun.com/ubuntu focal-updates/universe amd64 Packages [1153 kB]
Get:12 http://mirrors.aliyun.com/ubuntu focal-updates/restricted amd64 Packages [1178 kB]
Get:13 http://mirrors.aliyun.com/ubuntu focal-backports/main amd64 Packages [51.2 kB]
Get:14 http://mirrors.aliyun.com/ubuntu focal-backports/universe amd64 Packages [26.0 kB]
Get:15 http://mirrors.aliyun.com/ubuntu focal-security/restricted amd64 Packages [1104 kB]
Get:16 http://mirrors.aliyun.com/ubuntu focal-security/universe amd64 Packages [867 kB]
Get:17 http://mirrors.aliyun.com/ubuntu focal-security/multiverse amd64 Packages [25.8 kB]
Get:18 http://mirrors.aliyun.com/ubuntu focal-security/main amd64 Packages [1729 kB]
Fetched 21.7 MB in 2min 31s (144 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.

2 安装openssh。

root@19bafff6decc:/# apt install openssh-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  ca-certificates dbus distro-info-data dmsetup file gir1.2-glib-2.0 krb5-locales libapparmor1 libargon2-1 libbsd0
...
Created symlink /etc/systemd/system/multi-user.target.wants/networkd-dispatcher.service → /lib/systemd/system/networkd-dispatcher.service.
Setting up ssh-import-id (5.10-0ubuntu1) ...