Docker私有仓库搭建笔记（五）--配置HTTPS 弯路篇

为什么说是弯路篇，因为按文档来配置，一直启动不启来，然后不断的定位问题，查出原因。一度怀疑是harbor官方包有问题。

问题开始前先说下环境准备：

Centos 7.6:

harbor v1.8.2 版本。（前四章都用1.8.1）因怀 疑升到了 1.8.2 。

说下解压目录：本人解压在/home/hb下

为了配https 我选建一个证书存放目录： /home/hb/cert在里面放置了key和pem, 版本中启动时，用到的是crt格式的。但我只有key和pem.姑且就这个。

按官方配置说明步骤：

1. cd /home/hb/harbor  到 配置运行目录。

2. vi harbor.yml   后修改几个地方

hostname: hub.xxx.com    //自己对应的域名，没有话就用ip地址。

打开https, 

及设置证书和key的路径。

然后保存退出。

3.  执行 ./prepare   这个操作是大发师啊，一招就把所有配置都重新生成了。

    

[root@172-19-183-98 harbor]# ./prepare 
prepare base dir is set to /home/hb/harbor
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registry/root.crt
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/cert/server.crt
Clearing the configuration file: /config/cert/server.key
Clearing the configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir

4.  执行 ./install.sh    这个就是安装运行了(从执行来看，真心觉得第3步没必要)。如果你这样设置能成功运行。那么真的恭喜，至少我在centos 上没有成功。

    

[root@172-19-183-98 harbor]# ./install.sh 

[Step 0]: checking installation environment ...

Note: docker version: 18.09.7

Note: docker-compose version: 1.24.1

[Step 1]: loading Harbor images ...
Loaded image: goharbor/prepare:v1.8.2
Loaded image: goharbor/registry-photon:v2.7.1-patch-2819-v1.8.2
Loaded image: goharbor/notary-signer-photon:v0.6.1-v1.8.2
Loaded image: goharbor/chartmuseum-photon:v0.9.0-v1.8.2
Loaded image: goharbor/harbor-log:v1.8.2
Loaded image: goharbor/harbor-jobservice:v1.8.2
Loaded image: goharbor/redis-photon:v1.8.2
Loaded image: goharbor/clair-photon:v2.0.8-v1.8.2
Loaded image: goharbor/harbor-portal:v1.8.2
Loaded image: goharbor/harbor-core:v1.8.2
Loaded image: goharbor/nginx-photon:v1.8.2
Loaded image: goharbor/notary-server-photon:v0.6.1-v1.8.2
Loaded image: goharbor/harbor-db:v1.8.2
Loaded image: goharbor/harbor-registryctl:v1.8.2
Loaded image: goharbor/harbor-migrator:v1.8.2


[Step 2]: preparing environment ...
prepare base dir is set to /home/hb/harbor
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/cert/server.crt
Clearing the configuration file: /config/cert/server.key
Clearing the configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir


Note: stopping existing Harbor instance ...
Stopping nginx             ... done
Stopping harbor-portal     ... done
Stopping harbor-jobservice ... done
Stopping harbor-core       ... done
Stopping harbor-db         ... done
Stopping registryctl       ... done
Stopping registry          ... done
Stopping redis             ... done
Stopping harbor-log        ... done
Removing nginx             ... done
Removing harbor-portal     ... done
Removing harbor-jobservice ... done
Removing harbor-core       ... done
Removing harbor-db         ... done
Removing registryctl       ... done
Removing registry          ... done
Removing redis             ... done
Removing harbor-log        ... done
Removing network harbor_harbor


[Step 3]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating redis       ... done
Creating registryctl ... done
Creating harbor-db   ... done
Creating registry    ... done
Creating harbor-core ... done
Creating harbor-jobservice ... done
Creating harbor-portal     ... done
Creating nginx             ... done

✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at https://hub.xxx.com. 
For more details, please visit https://github.com/goharbor/harbor .

执行后从日志中显示是成功的。这个时候你看到提示https://hub.xxx.com成功了？真兴奋啊，马上把网址CP到浏览器上呗，结果，结果就是GG。啥也访问不了。好吧，排查问题开始。

很明显可以看到nginx 一直在重试。意思就是死活启不来呗。好，那nginx有什么问题，怎么样看日志呢？

仍然是在/home/hb/harbor目录下

iv docker-compose.yml

                         (图docker-compose.yml)

可以看到docker-compose中配置了日志依赖和挂裁的目录。为了看日志，我们先把最后那段logging:和依赖去除。

再次执行 docker-compose down -v 先全部停止，再docker-compose up -d.

然后执行

docker logs --tail=100 nginx    查看一下日志，发现：

2019/08/27 14:44:40 [emerg] 1#0: cannot load certificate "/etc/cert/server.crt": 
PEM_read_bio_X509_AUX() failed (SSL: error:25066067:DSO support 
routines:DLFCN_LOAD:could not load the shared library:filename(libz.so): libz.so: 
cannot open shared object file: No such file or directory error:25070067:DSO 
support routines:DSO_load:could not load the shared library error:0906D06C:PEM 
routines:PEM_read_bio:no start line)

nginx: [emerg] cannot load certificate "/etc/cert/server.crt": 
PEM_read_bio_X509_AUX() failed (SSL: error:25066067:DSO support 
routines:DLFCN_LOAD:could not load the shared library:filename(libz.so): libz.so: 
cannot open shared object file: No such file or directory error:25070067:DSO 
support routines:DSO_load:could not load the shared library error:0906D06C:PEM 
routines:PEM_read_bio:no start line)

可以看到，哎呀，路径找不到这个证书。好吧，看下这个是在那里配置的。

见前面的(图docker-compose.yml)可以看到volumes 中有指定source 和target,可以看到，我们的source配置的就是/home/hb/harbor中的harbor.yml中的证书路径。因为我没有crt文件，难道是这个原因生，好，手动改下。把crt改成pem.

    volumes:
      - ./common/config/nginx:/etc/nginx:z
      - type: bind
        source: /home/hb/cert
        target: /etc/cert/server.key
      - type: bind
        source: /home/hb/cert
        target: /etc/cert/server.pem

 这里只是改了docker run的，还要改nginx.conf,那这个 又在什么地方？看到./common/config/nginx么，说明应该是在harbor下的common下的，好，找到nginx.conf修改一下。

[root@172-19-183-98 harbor]# vi common/config/nginx/nginx.conf 

...
  server {
    listen 443 ssl;
#    server_name harbordomain.com;
    server_tokens off;
    # SSL
    ssl_certificate /etc/cert/server.pem; #改为pem,原来是crt的，因为我没有，所以用pem
    ssl_certificate_key /etc/cert/server.key;

....

修改完成后，执行docker-compose up -d

好像是正常？

nginx: [emerg] cannot load certificate "/etc/cert/server.pem": 
PEM_read_bio_X509_AUX() failed (SSL: error:25066067:DSO support 
routines:DLFCN_LOAD:could not load the shared library:filename(libz.so): libz.so: 
cannot open shared object file: No such file or directory error:25070067:DSO 
support routines:DSO_load:could not load the shared library error:0906D06C:PEM 
routines:PEM_read_bio:no start line)

查看好像还是一样。仍是找不到证书问题。

那好吧，出大招。修改:docker-compose.yml中的proxy节

...
proxy:
    image: goharbor/nginx-photon:v1.8.2
    container_name: nginx
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
      - NET_BIND_SERVICE
    volumes:
      - ./common/config/nginx:/etc/nginx:z
      - ./common/config/cert:/etc/nginx/cert
    networks:
      - harbor
    dns_search: .
    ports:
      - 80:80
      - 443:443
    depends_on:
      - postgresql
      - registry
      - core
      - portal
...

主要就是改了volumes增加了证书目录的挂载路径，把证书.key和.pem考到common/config/cert 的目录下。然后再修改一次nginx.conf中的证书目录

...
  server {
    listen 443 ssl;
#    server_name harbordomain.com;
    server_tokens off;
    # SSL
    # 因为挂载的是./common/config/cert，所以实际上这个/etc/nginx/cert访问的就是实际路径
    # ./common/config/cert中的文件。
    ssl_certificate /etc/nginx/cert/server.pem;
    ssl_certificate_key /etc/nginx/cert/server.key;
...

先执行docker-compose down -v

再次 docker-compose up -d

再后docker ps

然后在浏览器中进行访问https://hub.xxx.com

可以看到，能正常打开网页了。但好像使用上还有点问题吧。

不知是不是https的原因造成的浏览器访问有问题。

 

查到原因了，因为重新执行docker-compose down -v后，./common/config/cert中的server.key和server.pem被重置为0kb导至证书失效了。重新把这个证书copy一下，就好了。