Docker 快速实操笔记

概要

这个文章主要讲已经理解docker的前提下,快速利用docker安装各种组件的全集,所有的命令都已经测试过,作为新如果公司的大数据工程师是非常快速有效的

---

docker 安装

安装docker 
sudo yum install docker-ce docker-ce-cli containerd.io
安装docker compose
curl -L "https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

如果新机器,想要删除旧版本,切记,看清楚是否需要删除

 sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

非root用户想要用root下安装的docker怎么办？

sudo groupadd docker     #添加docker用户组
sudo gpasswd -a $USER docker     #将登陆用户加入到docker用户组中
newgrp docker     #更新用户组
docker ps    #测试docker命令是否可以使用sudo正常使用
scp -r /Users/jiangwenfeng/docker/hummingbird-docker root@centos7:/data/docker

也可以用比较简单的安装docker

yum install docker
docker -v  

替换阿里源

mkdir -p /etc/docker  && touch /etc/docker/daemon.json  
echo { "registry-mirrors": ["https://rvof9czj.mirror.aliyuncs.com"] } > /etc/docker/daemon.json

刷新
systemctl daemon-reload
重启
systemctl restart docker
查看是否生效
docker info|grep "Registry Mirrors" -A 1
输出
Registry Mirrors: https://rvof9czj.mirror.aliyuncs.com/

mysql 安装

版本可以自己选
docker search mysql

下载镜像
docker pull mysql:5.7

启动前构建本地目录用于存储mysql的日志和数据
mkdir -p /data/docker/data/mysql/log /data/docker/data/mysql/data

启动
docker run -d  -p 3308:3306 -v /data/docker/data/mysql/log:/var/log/mysql \ 
-v  /data/docker/data/mysql/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123456 mysql:5.7 \
--character-set-server=utf8mb4  --collation-server=utf8mb4_unicode_ci    --这个参数可以去参考mysql的配置任意指定

Redis

下载
docker pull redis
启动
docker run --name redis -v /data/docker/redis/data:/data -p 6379:6379 -d --restart=always redis:latest redis-server --appendonly yes --requirepass "redis" 
进入容器
docker exec -it  xxxx redis-cli

配置初始密码
auth redis 
config get requirepass
密码就是redis 

elasticsearch

版本可以自己选,去dockerhub的官网上都可以看到

非持久化方案

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.5.0
非持久化的方案
docker run -d --name es -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node"  \ 
docker.elastic.co/elasticsearch/elasticsearch:7.5.0

持久化方案
mkdir -p /data/elasticsearch/data /data/elasticsearch/logs

docker run --name es \
-p 9201:9200 -p 9301:9300 \
-v /data/elasticsearch/data:/var/lib/elasticsearch \
-v /data/elasticsearch/logs:/var/log/elasticsearch \
-d docker.elastic.co/elasticsearch/elasticsearch:7.5.0 \
-v /data/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml 

非必须

docker pull mobz/elasticsearch-head
docker run -d -p 9100:9100 --name elasticsearch-head elasticsearch-head

kafka && zookeeper

kafka是强依赖zookeeper的,所以这里一起介绍了

单机版

docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka

docker run -d --name zookeeper  -p 2181:2181 wurstmeister/zookeeper

docker run -d --name kafka \
--env KAFKA_ADVERTISED_HOST_NAME=localhost \
--env KAFKA_ZOOKEEPER_CONNECT=localhost:2181 \
--env KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \
--env KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
--env KAFKA_HEAP_OPTS="-Xmx256M -Xms128M" \
--net=host wurstmeister/kafka

单机版测试

单机测试:
docker exec -it kafka /bin/bash
cd /opt/kafka_2.12-2.4.0/ 
./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
./kafka-console-producer.sh --broker-list localhost:9092 --topic test
./kafka-topics.sh --list --zookeeper localhost:2181
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

集群版… 用dockercompose 的方式非常美好.但是我暂时忘记dockerfile的地址了,后面补充

kafka-manager

docker pull sheepkiller/kafka-manager

 docker pull solsson/kafka-manager:latest

docker run -d -p 9999:9000 -e  \ 
ZK_HOSTS=localhost:2181 solsson/kafka-manager

三. zookeeper

http://mirror.bit.edu.cn/apache/zookeeper/

RabbitMQ

docker pull rabbitmq
docker pull rabbitmq:management
docker run --name rabbitmq -d -p 15672:15672 -p 5672:5672 rabbitmq:management

pushgateway

docker pull prom/pushgateway

docker run -d -p 9091:9091 prom/pushgateway
验证:
http://localhost:9091/#

prometheus

docker pull prom/prometheus

docker run  -d \
  -p 9090:9090 \
  -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml  \
  prom/prometheus 

http://localhost:9090

grafana

docker pull grafana/grafana
docker run --rm -d \
  -p 3000:3000 \
  --name=grafana \
  -v /data/docker/grafana:$PWD/grafana-storage \
  grafana/grafana

clickhouse

docker pull yandex/clickhouse-server
docker pull yandex/clickhouse-client


docker  run --cpus 8 --memory 20G   --rm  -it   --name ck  \
-v `pwd`/data/db:/var/lib/clickhouse \
-v `pwd`/data/logs:/var/log/clickhouse-server   \
-v /etc/localtime:/etc/localtime   \
-v /usr/share/zoneinfo:/usr/share/zoneinfo \
-v `pwd`/clickhouse-server:/etc/clickhouse-server  \
-p8123:8123 -p9000:9000 -p9009:9009  docker.io/yandex/clickhouse-server



docker run -it --rm   yandex/clickhouse-client \
--user root--password 123456 -m --host hostname

spark

http://

spark.apache.org/downloads.html
http://mirror.bit.edu.cn/apache/spark/spark-2.4.5/spark-2.4.5-bin-hadoop2.7.tgz
hive
http://mirror.bit.edu.cn/apache/hive/

hbase

http://mirrors.tuna.tsinghua.edu.cn/apache/hbase/

kafka

http://kafka.apache.org/downloads
http://mirror.bit.edu.cn/apache/kafka/2.4.0/kafka_2.11-2.4.0.tgz
flink
https://downloads.apache.org/flink/flink-1.10.0/apache-flink-1.10.0.tar.gz

flume

http://flume.apache.org/download.html