Docker Registry搭建以及在K8S中使用

创建registry

1.拉取registry镜像:docker pull registry:2.7

[root@ ~]# docker pull registry:2.7
2.7: Pulling from library/registry
486039affc0a: Pull complete
ba51a3b098e6: Pull complete
8bb4c43d6c8e: Pull complete
6f5f453e5f2d: Pull complete
42bc10b72f42: Pull complete
Digest: sha256:7d081088e4bfd632a88e3f3bcd9e007ef44a796fddfe3261407a3f9f04abe1e7
Status: Downloaded newer image for registry:2.7
docker.io/library/registry:2.7

2. 查看下载的本地镜像:docker images

[root@ ~]# docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
registry                                  2.7                 708bc6af7e5e        3 months ago        25.8MB

3. 创建生成密码

[root@ ~]# cd /opt/
[root@ opt]# mkdir auth
[root@ opt]# echo "user:admin passwd:admin123" > htpasswd
[root@ opt]# docker run --entrypoint htpasswd registry:2.7 -Bbn admin admin123 > auth/htpasswd
[root@ opt]# cat auth/htpasswd
admin:$2y$05$6KftIJR6K.rEEg/0AU20vOTRbwvC88ngL6iDy.C2x65KaHLQ0oPI6

4. 在docker中运行registry，参数中指定后台运行，本地端口映射容器端口，设置了restart=always保持一直启动，挂载目录和设置环境变量

docker run -d -p 5000:5000 \
--restart=always --name registry \
-v /data/registry:/var/lib/registry \
-v /opt/auth:/auth \
-e "REGISTRY_AUTH=htpasswd" \
-e  "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
-e  REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd registry:2.7

5. docker ps 查看容器的运行情况
6. docker login 127.0.0.1:5000 登陆，会提示输入账户密码，然后提示https报错

docker login 127.0.0.1:5000
Username: admin
Password:
INFO[0007] Error logging in to v2 endpoint, trying next endpoint: Get https://127.0.0.1:5000/v2/: http: server gave HTTP response to HTTPS client
Get https://127.0.0.1:5000/v2/: http: server gave HTTP response to HTTPS client

7. docker客户端设置不需要https认证

cat /etc/docker/daemon.json
{
    "registry-mirrors":["https://hub-mirror.c.163.com/","http://f1361db2.m.daocloud.io"],
    "insecure-registries":["127.0.0.1:5000"],
    "metrics-addr" : "0.0.0.0:9323",
    "experimental" : true
}

8. 重新reload，启动一下docker，发现就可以从私有镜像仓库中拉取变量了

systemctl daemon-reload
systemctl restart docker

在 K8S 中使用私有镜像

1. 使用kubectl 创建一个secret

kubectl create secret docker-registry my-secret-26 --docker-server=127.0.0.1:5000 --docker-username=admin --docker-password=admin123 --docker-email=caoke@qq.com

2. 使用 kebuctl get secret 查看使用创建成功

[root@cn ~]# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-skmls   kubernetes.io/service-account-token   3      10d
my-registry-26        kubernetes.io/dockerconfigjson        1      17h
my-secret-26          kubernetes.io/dockerconfigjson        1      16h

3. 在提交给k8s的yml文件中指定pull的配置
   Pod 示例

[root@c.cn dockerdemo]# cat logstash-demo.yml
apiVersion: v1
kind: Pod
metadata:
  name: logstash-demo
spec:
  containers:
  - name: logstash
    image: 127.0.0.1:5000/logstash:6.2.4  #使用私服的地址
  imagePullSecrets:
  - name: my-secret-26  #配置的secret

Deploy示例

[root@cyn dockerdemo]# cat logstash-deploy.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: logstash-deployment
  labels:
    app: logstash-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: logstash-deployment
  template:
    metadata:
      labels:
        app: logstash-deployment
    spec:
      imagePullSecrets:
      - name: my-secret-26
      containers:
      - name: logstash-deployment
        image: 127.0.0.1:5000/logstash:6.2.4

4. 重要： docker registry 需要https，而我们没有证书，所以需要再docker 客户端加上一段配置 “insecure-registries”:[“127.0.0.1:5000”]

[root@cycn dockerdemo]# cat /etc/docker/daemon.json
{
    "registry-mirrors":["https://hub-mirror.c.163.com/","http://f1361db2.m.daocloud.io"],
    "insecure-registries":["127.0.0.1:5000"],
    "metrics-addr" : "0.0.0.0:9323",
    "experimental" : true
}

5. 一定是要在所有的k8s节点都给配上，我第一次只是在master上面配置了，结果提交服务后怎么也启动不了，在我咨询了同事后，提醒我在所有的worker节点都配置上，一下子就pull下来了，太开心了，困扰了我大半天的问题。原本就想既然本地docker pull 都能成功了，为什么到k8s里面拉不下来呢，k8s底层也用的docker呀，原来是因为在worker点节执行，worker节点没有配置的原因。
   附上报错信息：

Error logging in to v2 endpoint, trying next endpoint: Get https://127.0.0.1:5000/v2/: http: server gave HTTP response to HTTPS client
Get https://127.0.0.1:5000/v2/: http: server gave HTTP response to HTTPS client

环境

Docker

 docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.39 (downgraded from 1.40)
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:27:04 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 17:26:28 2019
  OS/Arch:          linux/amd64
  Experimental:     true

Docker registry 2.7
kubelet

kubelet --version
Kubernetes v1.15.4

kubectl

kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.4", GitCommit:"67d2fcf276fcd9cf743ad4be9a9ef5828adc082f", GitTreeState:"clean", BuildDate:"2019-09-18T14:51:13Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.4", GitCommit:"67d2fcf276fcd9cf743ad4be9a9ef5828adc082f", GitTreeState:"clean", BuildDate:"2019-09-18T14:41:55Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

kubeadm

kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.4", GitCommit:"67d2fcf276fcd9cf743ad4be9a9ef5828adc082f", GitTreeState:"clean", BuildDate:"2019-09-18T14:48:18Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

OS

 uname -a
Linux cyn 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@cycn dockerdemo]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)