在Linux中搭建harbor私有仓库

一、环境准备

Harbor的所有服务组件都是在Docker中部署的，所以官方安装使用Docker-compose快速部署，所以需要安装Docker、Docker-compose。由于Harbor是基于Docker Registry V2版本，所以就要求Docker版本不小于1.10.0，Docker-compose版本不小于1.6.0。

1、安装启动docker

安装所需的包。yum-utils提供了yum-config-manager 效用，并device-mapper-persistent-data和lvm2由需要 devicemapper存储驱动程序。

[root@node2 ~]#   yum install -y yum-utils device-mapper-persistent-data lvm2

设置稳定存储库

[root@node2 ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

安装Docker CE

[root@node2 ~]# yum install -y docker-ce docker-ce-cli containerd.io

2、安装Docker-compose

下载指定版本的docker-compose

[root@node2 ~]# curl -L https://github.com/docker/compose/releases/download/1.13.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

如果下载的比较慢的话可以使用下面这个

[root@node2 ~]#curl -L "https://get.daocloud.io/docker/compose/releases/download/1.13.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

对二进制文件赋可执行权限

root@node2 ~]# chmod +x /usr/local/bin/docker-compose

测试下docker-compose是否安装成功

[root@node2 ~]# docker-compose version
docker-compose version 1.13.0, build 1719ceb
docker-py version: 2.2.1
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016

二、Harbor服务搭建及启动

1、下载Harbor安装文件

root@localhost ~]# mkdir -p /opt/harbor
[root@localhost ~]# cd /opt/harbor/
[root@localhost harbor]# yum -y install wget
[root@localhost harbor]# wget https://github.com/vmware/harbor/releases/download/v1.1.2/harbor-online-installer-v1.1.2.tgz
[root@localhost harbor]# ls
harbor-online-installer-v1.1.2.tgz
[root@localhost harbor]# tar -zxf harbor-online-installer-v1.1.2.tgz 

2、配置Harbor

[root@localhost harbor]# ls
harbor  harbor-online-installer-v1.1.2.tgz
[root@localhost harbor]# cd harbor
[root@localhost harbor]# ls
common  docker-compose.notary.yml  docker-compose.yml  harbor_1_1_0_template  harbor.cfg  install.sh  LICENSE  NOTICE  prepare  upgrade
[root@localhost harbor]# vi harbor.cfg 

• 配置文件harbor.cfg详解：

# hostname设置访问地址，可以使用ip、域名，不可以设置为127.0.0.1或localhost
hostname = https://hub.whcloud.com

# 访问协议，默认是http，也可以设置https，如果设置https，则nginx ssl需要设置on
ui_url_protocol = https

# mysql数据库root用户默认密码root123，实际使用时修改下
db_password = 123456

max_job_workers = 3 

customize_crt = on

ssl_cert = /data/cert/server.crt
ssl_cert_key = /data/cert/server.key

secretkey_path = /data

admiral_url = NA
# 邮件设置，发送重置密码邮件时使用
email_identity = 

email_server = smtp.mydomain.com
email_server_port = 25
email_username = sample_admin@mydomain.com
email_password = abc
email_from = admin <sample_admin@mydomain.com>
email_ssl = false

# 启动Harbor后，管理员UI登录的密码，默认是Harbor12345
harbor_admin_password = 123456

# 认证方式，这里支持多种认证方式，如LADP、本次存储、数据库认证。默认是db_auth，mysql数据库认证
auth_mode = db_auth

# LDAP认证时配置项
ldap_url = ldaps://ldap.mydomain.com
#ldap_searchdn = uid=searchuser,ou=people,dc=mydomain,dc=com
#ldap_search_pwd = password
ldap_basedn = ou=people,dc=mydomain,dc=com
#ldap_filter = (objectClass=person)
ldap_uid = uid
ldap_scope = 3
ldap_timeout = 5

# 是否开启自注册
self_registration = on

# token有效时间，默认30分钟
token_expiration = 30

# 用户创建项目权限控制，默认是everyone（所有人），也可以设置为adminonly（只能管理员）
project_creation_restriction = everyone

verify_remote_cert = on

3、设置https,配置私钥，生成证书

[root@ecs-431f-0003 cert]# mkdir -p  /data/cert
[root@ecs-431f-0003 cert]# cd /data/cert
# 需要输入两次密码 (root好记)
[root@ecs-431f-0003 cert]# openssl genrsa -des3 -out server.key 2048 
Generating RSA private key, 2048 bit long modulus (2 primes)
...........................+++++
..................+++++
e is 65537 (0x010001)
Enter pass phrase for server.key:
Verifying - Enter pass phrase for server.key:

[root@ecs-431f-0003 cert]# openssl req -new -key server.key -out server.csr
Enter pass phrase for server.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:BJ
Locality Name (eg, city) [Default City]:BJ
Organization Name (eg, company) [Default Company Ltd]:WEU
Organizational Unit Name (eg, section) []:WEIHU
Common Name (eg, your name or your server's hostname) []:119.3.177.131
Email Address []:2820524218@qq.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

[root@ecs-431f-0003 cert]# cp server.key server.key.org
#退出:docker引导nginx，需要退出私钥，不包含密码 输入上面的密码
[root@ecs-431f-0003 cert]# openssl rsa -in server.key.org -out server.key
Enter pass phrase for server.key.org:
writing RSA key

#添加证书
[root@ecs-431f-0003 cert]# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
Signature ok
subject=C = CN, ST = BJ, L = BJ, O = WEU, OU = WEIHU, CN = 119.3.177.131, emailAddress = 2820524218@qq.com
Getting Private key

# 添加权限
[root@ecs-431f-0003 cert]# chmod -R 777 /data/cert

• 目录如下：

• vi /etc/docker/daemon.json文件,如果没有就创建一个。

{
  "insecure-registries": ["https://hub.whcloud.com"]
}

• 在Linux中的host文件中进行一个域名的映射。

[root@node1 docker]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.198.131 hub.whcloud.com

• 在windows中的hosts中添加域名的映射。添加如下一行。

C:\Windows\System32\drivers\etc

4、启动Harbor
修改完配置文件后，在的当前目录执行./install.sh，Harbor服务就会根据当期目录下的docker-compose.yml开始下载依赖的镜像，检测并按照顺序依次启动各个服务。

# 前面的操作需要重新启动一下docker
[root@node2 harbor]# systemctl restart docker
[root@node2 harbor]#./install
...
[Step 3]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... 
Creating harbor-log ... done
Creating registry ... 
Creating harbor-adminserver ... 
Creating harbor-db ... 
Creating harbor-adminserver
Creating registry
Creating registry ... done
Creating harbor-ui ... 
Creating harbor-ui ... done
Creating nginx ... 
Creating harbor-jobservice ... 
Creating nginx
Creating nginx ... done

✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at https://hub.whcloud.com. 
For more details, please visit https://github.com/vmware/harbor .

三 、harbor仓库使用

1、登录Web Harbor，密码是上面配置文件中设置的

• 2、新建一个项目

我们新建一个名称为whcloud的项目，设置不公开。当项目设为公开后，任何人都有此项目下镜像的读权限。命令行用户不需要docker login就可以拉取此项目下的镜像

新建项目后，使用admin用户提交本地nginx镜像到Harbor仓库

• 3、admin登录

[root@node2 harbor]# docker login https://hub.whcloud.com
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

• 登录中遇到的问题
  ① 错误原因：生成证书时没有重启docker

root@node2 harbor]# docker login https://hub.whcloud.com
Username: admin
Password: 
Error response from daemon: Get https://hub.whcloud.com/v2/: x509: certificate signed by unknown authority
#启动一下docker
[root@node2 harbor]# sudo systemctl restart docker

**②仓库中没有新建一个项目，需要新建一个项目 **

[root@ananzj harbor]# docker login http://39.106.254.42 -u admin
Password: 
Error response from daemon: Get http://39.106.254.42/v2/: unauthorized: authentication required

创建完项目再登录(在阿里云下进行登录)

[root@ananzj harbor]# docker login http://39.106.254.42 -u admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

③ 拒绝连接：可能就是harbor的问题

[root@node2 harbor]# docker login https://hub.whcloud.com
Username: admin
Password: 
Error response from daemon: Get http://hub.whcloud.com/v2/: dial tcp 192.168.198.131:80: connect: connection refused

解决办法,在harbor目录下执行命令。

[root@node2 harbor]# docker-compose down -v
Stopping nginx ... done
Stopping harbor-ui ... done
Stopping harbor-adminserver ... done
Stopping harbor-log ... done
Removing harbor-jobservice ... done
Removing nginx ... done
Removing harbor-ui ... done
Removing harbor-db ... done
Removing registry ... done
Removing harbor-adminserver ... done
Removing harbor-log ... done
Removing network harbor_harbor
[root@node2 harbor]# docker-compose up -d
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... 
Creating harbor-log ... done
Creating registry ... 
Creating harbor-adminserver ... 
Creating harbor-db ... 
Creating harbor-db
Creating registry
Creating registry ... done
Creating harbor-ui ... 
Creating harbor-ui ... done
Creating nginx ... 
Creating harbor-jobservice ... 
Creating nginx
Creating nginx ... done
[root@node2 harbor]# docker login https://hub.whcloud.com
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
# 登录成功

• 4. 给镜像打tag

[root@node2 harbor]# docker tag registry.cn-hangzhou.aliyuncs.com/anzhijie/anapp:v2 hub.whcloud.com/harbor/tomcat:9.0
[root@node2 harbor]# docker images
REPOSITORY                                                                TAG                 IMAGE ID            CREATED             SIZE
registry.cn-hangzhou.aliyuncs.com/anzhijie/bolingcavalry                  v2.0                9448aa871630        9 days ago          44MB
mysql                                                                     5.6                 eb0e825dc3cf        3 weeks ago         303MB
hub.whcloud.com/harbor/tomcat                                             9.0                 7e78ebfe09e4        3 weeks ago         604MB

• 5、推送到harbor仓库

[root@node2 harbor]# docker push hub.whcloud.com/harbor/tomcat:9.0
The push refers to repository [hub.whcloud.com/harbor/tomcat]
73ead83820ce: Pushed 
f54b866f0424: Pushed 
2653d992f4ef: Pushed 
9.0: digest: sha256:a728fd02694be39a0669897ed060dc386db06a4d235205e45739cd47001896f1 size: 953

• 6、在其他的服务器pull镜像

在node1节点执行，vi /etc/docker/daemon.json文件,如果没有就创建一个。

{
  "insecure-registries": ["https://hub.whcloud.com"]
}

在node1节点同时添加一个域名的映射。

[root@node1 docker]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.198.131 hub.whcloud.com

使用docker pull 拉取harbor中的镜像

[root@node1 docker]# docker pull hub.whcloud.com/harbor/commonannotation:v1
v1: Pulling from harbor/commonannotation
7a0437f04f83: Already exists 
67b368d51978: Already exists 
7231fa79af00: Already exists 
e9d560797a9c: Pull complete 
93866b70b134: Pull complete 
Digest: sha256:26c3b9fbc1c649930c552527c45633913648ad561e7b23a0a514c322053c722a
Status: Downloaded newer image for hub.whcloud.com/harbor/commonannotation:v1
[root@node1 docker]# docker images
[root@node1 docker]# docker images
REPOSITORY                                           TAG                 IMAGE ID            CREATED                  SIZE
hub.whcloud.com/harbor/commonannotation              v1                  3aebc6206c1f        Less than a second ago   604MB
prom/node-exporter                                   latest              0fafea149859        5 days ago               21.2MB

四、连接docker客户端

vi /lib/systemd/system/docker.service ExecStart后面的参数负责

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
#ExecStart=/usr/bin/dockerd 
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix://var/run/docker.sock --insecure-registry hub.whcloud.com
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target

测试:成功监听到2375端口

[root@node1 systemd]# netstat -ntlp |grep dockerd
tcp6       0      0 :::2375                 :::*                    LISTEN      8029/dockerd        

相当于命令docker images

[root@node1 systemd]# docker -H 192.168.198.132:2375 images
REPOSITORY                                           TAG                 IMAGE ID            CREATED             SIZE
hub.whcloud.com/harbor/commonannotation              v1                  3aebc6206c1f        8 hours ago         604MB
prom/node-exporter                                   latest              0fafea149859        6 days ago          21.2MB
registry.cn-hangzhou.aliyuncs.com/anzhijie/anapp     v2                  7e78ebfe09e4        3 weeks ago         604MB
prom/node-exporter                                   <none>              b8a614cef113        3 weeks ago         21.2MB
quay.io/coreos/flannel                               v0.14.0             8522d622299c        2 months ago        67.9MB
registry.aliyuncs.com/google_containers/kube-proxy   v1.18.0             43940c34f24f        16 months ago       117MB
registry.aliyuncs.com/google_containers/pause        3.2                 80d28bedfe5d        18 months ago       683kB
vmware/harbor-log                                    v1.2.0              c7887347f435        3 years ago         200MB
vmware/harbor-jobservice                             v1.2.0              1fb18427db11        3 years ago         164MB
vmware/harbor-ui                                     v1.2.0              b7069ac3bd4b        3 years ago         178MB
vmware/harbor-adminserver                            v1.2.0              a18331f0c1ae        3 years ago         142MB
vmware/harbor-db                                     v1.2.0              deb8033b1c86        3 years ago         329MB
vmware/registry                                      2.6.2-photon        5d9100e4350e        3 years ago         173MB
vmware/nginx-photon                                  1.11.13             285492ff20d6        4 years ago         147MB
tomcat                                               8.0.18-jre8         ad9a5d210c94        6 years ago         492MB