在Docker上部署Hortonworks沙箱

操作环境

在Docker上部署Hortonworks沙箱，源文档中有l在inux、mac及windows上搭建的详细指导，此文档仅记录自己在mac本地搭建的过程，其他操作系统的搭建请参见源文档：https://www.cloudera.com/tutorials/sandbox-deployment-and-install-guide/3.html

系统要求

• mac 内存16G以上
• 安装Docker 17.09 或更新的版本
• Docker需要配置10G以上的内存

安装步骤

步骤1：下载安装脚本

$ wget --no-check-certificate https://archive.cloudera.com/hwx-sandbox/hdp/hdp-3.0.1/HDP_3.0.1_docker-deploy-scripts_18120587fc7fb.zip

步骤2：解压缩安装脚本

$ unzip unzip HDP_3.0.1_docker-deploy-scripts_18120587fc7fb.zip 

步骤3：执行安装脚本

执行如下命令，执行后会开始拉取docker镜像，需要下载29G左右的数据，需要耐心等待。

$ sh docker-deploy-hdp30.sh

步骤4：查看是否安装成功

数据下载成功后会自动启动两个docker服务，使用docker ps查看，如下所示，即是安装成功，

93030a161564   hortonworks/sandbox-proxy:1.0   "nginx -g 'daemon of…"   7 minutes ago   Up 22 seconds   0.0.0.0:1080->1080/tcp, :::1080->1080/tcp, 0.0.0.0:1100->1100/tcp, :::1100->1100/tcp, 0.0.0.0:1111->1111/tcp, :::1111->1111/tcp, 0.0.0.0:1988->1988/tcp, :::1988->1988/tcp, 0.0.0.0:2100->2100/tcp, :::2100->2100/tcp, 0.0.0.0:2181-2182->2181-2182/tcp, :::2181-2182->2181-2182/tcp, 0.0.0.0:2201-2202->2201-2202/tcp, :::2201-2202->2201-2202/tcp, 0.0.0.0:2222->2222/tcp, :::2222->2222/tcp, 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp, 0.0.0.0:4040->4040/tcp, :::4040->4040/tcp, 0.0.0.0:4200->4200/tcp, :::4200->4200/tcp, 0.0.0.0:4242->4242/tcp, :::4242->4242/tcp, 0.0.0.0:4557->4557/tcp, :::4557->4557/tcp, 0.0.0.0:5007->5007/tcp, :::5007->5007/tcp, 0.0.0.0:5011->5011/tcp, :::5011->5011/tcp, 0.0.0.0:6001->6001/tcp, :::6001->6001/tcp, 0.0.0.0:6003->6003/tcp, :::6003->6003/tcp, 0.0.0.0:6008->6008/tcp, :::6008->6008/tcp, 0.0.0.0:6080->6080/tcp, :::6080->6080/tcp, 0.0.0.0:6188->6188/tcp, :::6188->6188/tcp, 0.0.0.0:6627->6627/tcp, :::6627->6627/tcp, 0.0.0.0:6667-6668->6667-6668/tcp, :::6667-6668->6667-6668/tcp, 0.0.0.0:7777->7777/tcp, :::7777->7777/tcp, 0.0.0.0:7788->7788/tcp, :::7788->7788/tcp, 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:8005->8005/tcp, :::8005->8005/tcp, 0.0.0.0:8020->8020/tcp, :::8020->8020/tcp, 0.0.0.0:8032->8032/tcp, :::8032->8032/tcp, 0.0.0.0:8040->8040/tcp, :::8040->8040/tcp, 0.0.0.0:8042->8042/tcp, :::8042->8042/tcp, 0.0.0.0:8080-8082->8080-8082/tcp, :::8080-8082->8080-8082/tcp, 0.0.0.0:8086->8086/tcp, :::8086->8086/tcp, 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp, 0.0.0.0:8090-8091->8090-8091/tcp, :::8090-8091->8090-8091/tcp, 0.0.0.0:8188->8188/tcp, :::8188->8188/tcp, 0.0.0.0:8198->8198/tcp, :::8198->8198/tcp, 0.0.0.0:8443->8443/tcp, :::8443->8443/tcp, 0.0.0.0:8585->8585/tcp, :::8585->8585/tcp, 0.0.0.0:8744->8744/tcp, :::8744->8744/tcp, 0.0.0.0:8765->8765/tcp, :::8765->8765/tcp, 0.0.0.0:8886->8886/tcp, :::8886->8886/tcp, 0.0.0.0:8888-8889->8888-8889/tcp, :::8888-8889->8888-8889/tcp, 0.0.0.0:8983->8983/tcp, :::8983->8983/tcp, 0.0.0.0:8993->8993/tcp, :::8993->8993/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:9088-9091->9088-9091/tcp, :::9088-9091->9088-9091/tcp, 0.0.0.0:9995-9996->9995-9996/tcp, :::9995-9996->9995-9996/tcp, 0.0.0.0:10000-10002->10000-10002/tcp, :::10000-10002->10000-10002/tcp, 0.0.0.0:10015-10016->10015-10016/tcp, :::10015-10016->10015-10016/tcp, 0.0.0.0:10500->10500/tcp, :::10500->10500/tcp, 0.0.0.0:10502->10502/tcp, :::10502->10502/tcp, 0.0.0.0:11000->11000/tcp, :::11000->11000/tcp, 0.0.0.0:12049->12049/tcp, :::12049->12049/tcp, 0.0.0.0:12200->12200/tcp, :::12200->12200/tcp, 0.0.0.0:15000->15000/tcp, :::15000->15000/tcp, 0.0.0.0:15002->15002/tcp, :::15002->15002/tcp, 0.0.0.0:15500->15500/tcp, :::15500->15500/tcp, 0.0.0.0:16000->16000/tcp, :::16000->16000/tcp, 0.0.0.0:16010->16010/tcp, :::16010->16010/tcp, 0.0.0.0:16020->16020/tcp, :::16020->16020/tcp, 0.0.0.0:16030->16030/tcp, :::16030->16030/tcp, 0.0.0.0:18080-18081->18080-18081/tcp, :::18080-18081->18080-18081/tcp, 0.0.0.0:19888->19888/tcp, :::19888->19888/tcp, 0.0.0.0:21000->21000/tcp, :::21000->21000/tcp, 0.0.0.0:30800->30800/tcp, :::30800->30800/tcp, 0.0.0.0:33553->33553/tcp, :::33553->33553/tcp, 0.0.0.0:39419->39419/tcp, :::39419->39419/tcp, 0.0.0.0:42111->42111/tcp, :::42111->42111/tcp, 0.0.0.0:50070->50070/tcp, :::50070->50070/tcp, 0.0.0.0:50075->50075/tcp, :::50075->50075/tcp, 0.0.0.0:50079->50079/tcp, :::50079->50079/tcp, 0.0.0.0:50095->50095/tcp, :::50095->50095/tcp, 0.0.0.0:50111->50111/tcp, :::50111->50111/tcp, 0.0.0.0:60000->60000/tcp, :::60000->60000/tcp, 0.0.0.0:60080->60080/tcp, :::60080->60080/tcp, 0.0.0.0:61080->61080/tcp, :::61080->61080/tcp, 80/tcp, 0.0.0.0:61888->61888/tcp, :::61888->61888/tcp   sandbox-proxy
937e2c24ad66   hortonworks/sandbox-hdp:3.0.1   "/usr/sbin/init"         22 hours ago    Up 4 seconds    22/tcp, 4200/tcp, 8080/tcp                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              sandbox-hdp

Sandbox管理

停止sandbox

$ docker stop sandbox-hdp
$ docker stop sandbox-proxy

启动sandbox

$ docker start sandbox-hdp
$ docker start sandbox-proxy

删除sandbox

$ docker stop sandbox-hdp
$ docker stop sandbox-proxy
$ docker rm sandbox-hdp
$ docker rm sandbox-proxy

Sandbox的使用

UI

• 通过localhost:8080 进入ambari的管理页面，如下所示：
  
  关于登录Ambari的登陆密码可以参考https://www.cloudera.com/tutorials/learning-the-ropes-of-the-hdp-sandbox.html这个页面获取，根据不同用途可以选择不同的用户：
  |用户 |角色 |密码|
  |–|--|–|
  |admin |Ambari Admin |使用ambari-admin-password-reset命令初始化获得|
  |maria_dev |Spark and SQL Developer| maria_dev|
  |raj_ops |Hadoop Warehouse Operator |raj_ops|
  |holger_gov |Data Steward |holger_gov|
  |amy_ds |Data Scientist| amy_ds|

CLI

• docker exec -it sandbox-hdp bash # 进入docker服务，同在linux上的使用方式

参考文档

1. https://blog.csdn.net/chenchitan/article/details/113727361
2. https://www.cloudera.com/tutorials/sandbox-deployment-and-install-guide/3.html
3. https://www.cloudera.com/tutorials/learning-the-ropes-of-the-hdp-sandbox.html