概念
OpenShift可访问的Image Registy包括其自身内部的Image Registry和其外部的Image Registry,而外部的Image Registry又可分为运行在远端的Image Registry,以及为了加速远端访问而运行在本地(或近地的)Mirror Image Registry。Mirror Image Registry除了可以用来加速访问外部Image外,我们在离线安装OpenShift的时候也会用到它。
以上OpenShift用到的Image Registry配置是以对象的方式存放在Etcd中。当OpenShift的Machine Config Operator发现这些对象变化后会自动将其推送到每个Node并写入 /etc/containers/registries.conf 文件,OpenShift然后会重启对应的Node使配置生效。
配置OpenShift的Image Registry
OpenShift 4把它能访问到的外部Registry的配置信息是保存在“image.config.openshift.io/cluster”对象中。当OpenShift的Machine Config Operator(MCO)发现image.config.openshift.io配置变化后,会将最新配置对送到所有Node并写入 /etc/containers/registries.conf 文件,然后重启对应的Node使配置生效。
OpenShift使用一个名为cluster的image.config.openshift.io对象管理其能访问到的Image Registry目标和策略。
$ oc get image.config.openshift.io
NAME AGE
cluster 12d
查看OpenShift可以访问到Registry的配置,其中“External Registry Hostnames”和“Internal Registry Hostname”定义了外部和内部Registry。
$ oc describe image.config.openshift.io cluster
Name: cluster
Namespace:
Labels: <none>
Annotations: release.openshift.io/create-only: true
API Version: config.openshift.io/v1
Kind: Image
Metadata:
Creation Timestamp: 2020-07-18T09:54:27Z
Generation: 1
Resource Version: 25273
Self Link: /apis/config.openshift.io/v1/images/cluster
UID: c94ebed4-684a-4c98-86aa-658a8685ef79
Spec:
Status:
External Registry Hostnames:
default-route-openshift-image-registry.apps.cluster-beijing-959a.beijing-959a.example.opentlc.com
Internal Registry Hostname: image-registry.openshift-image-registry.svc:5000
Events: <none>
上面只是Registry的配置信息,其中OpenShift内部的Registry是通过以下的Cluster Operator运行的。
$ oc get co image-registry
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE
image-registry 4.5.4 True False False 2d2h
如果要修改OpenShift可访问的Registry目标,可先编辑image.config.openshift.io/cluster对象:
$ oc edit image.config.openshift.io/cluster
增加“registrySources”部分内容。
spec:
registrySources:
insecureRegistries:
- ocr.example.com
allowedRegistries:
- quay.io
- ocr.example.com
保存后OpenShift会自动通过MCO将最新的配置同步到各个节点。可以从Node的Events查看节点的重启过程。
如果需要查看一个Node的Registry配置,可先通过debug进入该Node,然后查看/etc/containers/registries.conf(该文件是podman可以访问的registry的配置),确认最新的Registry配置已经加进去了。(如果使用非完整名称的Image,podman会去由“unqualified-search-registries”定义的Registry查找该镜像)
$ oc debug node/<NODE-NAME>
sh-4.2# chroot /host
sh-4.2# cat /etc/containers/registries.conf
unqualified-search-registries = ['registry.access.redhat.com', 'docker.io']
[[registry]]
prefix = ""
location = "ocr.example.com"
insecure = true
[[registry]]
prefix = ""
location = "quay.io"
allowed = true
配置OpenShift的Mirror Image Registry
在一个受限的网络中(或访问远程Image Registry的网速比较慢),可以为OpenShift使用的外部Image Registry建一个Mirror,以加速OpenShift访问远程Registry上Image。Registry Mirror可以是运行在自己本地(或区域)的Docker Distribution、Quay等提供的Registry服务。
Image Registry Mirror的配置信息是保存在OpenShift集群范围的ImageContentSourcePolicy对象。当OpenShift的Machine Config Operator(MCO)发现ImageContentSourcePolicy配置变化后,会将最新配置对送到所有Node并写入 /etc/containers/registries.conf 文件,然后重启对应的Node使配置生效。
注意:本文只对说明如何将Mirror Image Registry的配置加到OpenShift中,而部署一套Mirror Image Registry可参见《OpenShift 4 - 部署Mirror Registry》。
- 创建如下内容的registryrepomirror.yaml文件,其中为“registry.access.redhat.com/ubi8/ubi-minimal”镜像提供了2个Mirror,一个是“ example.io/example/ubi-minimal”,另一个是“example.com/example/ubi-minimal”,OpenShift会按照顺序访问Mirror的。
apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
name: ubi8repo
spec:
repositoryDigestMirrors:
- mirrors:
- example.io/example/ubi-minimal
source: registry.access.redhat.com/ubi8/ubi-minimal
- mirrors:
- example.com/example/ubi-minimal
source: registry.access.redhat.com/ubi8/ubi-minimal
- 创建ImageContentSourcePolicy对象。
$ oc create -f registryrepomirror.yaml
- 查看Node的状态,确认Node会重启以使得配置生效。
$ oc get node
NAME STATUS ROLES AGE VERSION
ip-10-0-131-190.ap-southeast-1.compute.internal Ready master 13d v1.18.3+012b3ec
ip-10-0-141-25.ap-southeast-1.compute.internal Readyy,SchedulingDisabled worker 13d v1.18.3+012b3ec
ip-10-0-165-133.ap-southeast-1.compute.internal Ready master 13d v1.18.3+012b3ec
ip-10-0-174-85.ap-southeast-1.compute.internal Ready worker 13d v1.18.3+012b3ec
ip-10-0-216-155.ap-southeast-1.compute.internal Ready master 13d v1.18.3+012b3ec
- 确认配置已经更新,有2个registry.mirror的配置。
$ oc debug node <NODE-NAME>
sh-4.2# chroot /host
sh-4.4# cat /etc/containers/registries.conf
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"]
[[registry]]
prefix = ""
location = "registry.access.redhat.com/ubi8/ubi-minimal"
mirror-by-digest-only = true
[[registry.mirror]]
location = "example.com/example/ubi-minimal"
[[registry.mirror]]
location = "example.io/example/ubi-minimal"
[[registry]]
prefix = ""
location = "ocr.example.com"
insecure = true
[[registry]]
prefix = ""
location = "untrusted.com"
blocked = true
- 在Node内部发起对远程镜像“registry.access.redhat.com/ubi8/ubi-minimal”的访问,通过日志确认系统是从Mirror中获得该Image的。
sh-4.2# podman pull --log-level=debug registry.access.redhat.com/ubi8/ubi-minimal
参考
- https://computingforgeeks.com/how-to-setup-red-hat-quay-registry-on-centos-rhel/
- https://access.redhat.com/documentation/zh-cn/openshift_container_platform/4.7/html-single/images/index#installation-restricted-network-samples_samples-operator-alt-registry
- https://access.redhat.com/documentation/zh-cn/openshift_container_platform/4.7/html-single/images/index#images-configuration-parameters_image-configuration
- https://computingforgeeks.com/allow-insecure-registries-in-openshift-okd-4-cluster/
- https://rhthsa.github.io/openshift-demo/build-with-oc.html
- https://docs.openshift.com/container-platform/4.7/post_installation_configuration/preparing-for-users.html#images-configuration-shortname_post-install-preparing-for-users
- https://www.redhat.com/sysadmin/manage-container-registries