OpenShift 4 - 用Compliance Operator对OpenShift进行安全合规扫描（视频）

dawnsky.liu

已于 2024-06-27 20:14:31 修改

阅读量1.3k

点赞数

分类专栏：安全 OpenShift 4 文章标签：安全安全漏洞 openshift

于 2021-06-13 17:59:17 首次发布

本文链接：https://blog.csdn.net/weixin_43902588/article/details/115362493

版权

OpenShift 4 同时被 2 个专栏收录

272 篇文章 81 订阅

订阅专栏

安全

76 篇文章 6 订阅

订阅专栏

《OpenShift 4.x HOL教程汇总》
说明：本文已经在 OpenShift 4.15 环境中验证

本文介绍如何在 OpenShift 中进行安全合规扫描以及对违规项目进行自动修复。

安装Compliance Operator

进入OpenShift控制台OperatorHub，找到“Compliance Operator”点击进入。
接受缺省配置，安装“Compliance Operator”。缺省会将“Compliance Operator”安装到“openshift-compliance”项目中。
安装完进入“Compliance Operator”。

查看Compliance Operator相关对象

查看 profilebundle 对象，说明 profile 有以下 2 类，可以通过 label 区分它们。

$ oc get profilebundle.compliance -n openshift-compliance
NAME     CONTENTIMAGE                                                                                                                     CONTENTFILE         STATUS
ocp4     registry.redhat.io/openshift4/compliance-content-rhel8@sha256:4529b9bb32c1846a38e38363fa872713b1c1e6b26b34d887813432f97cff368c   ssg-ocp4-ds.xml     VALID
rhcos4   registry.redhat.io/openshift4/compliance-content-rhel8@sha256:4529b9bb32c1846a38e38363fa872713b1c1e6b26b34d887813432f97cff368c   ssg-rhcos4-ds.xml   VALID

查看通过Compliance Operator安装的所有Profile对象。其中 ocp4-xxx 是描述运行在 OpenShift 上相关的合规 Profile；ocp4-xxx-node 是描述运行在主机节点上的和 OpenShift 服务相关的合规 Profile；rhcos4-xxx 是描述节点 RHCOS 操作系统层面相关的合规 Profile。而 xxx-high 和 xxx-moderate 是描述该合规 Profile 的级别为“高级和中级”。此外按照不同合规适用领域，这些 Profile 又分为 “cis”、“e8”、“merc”和“pci”，可查看 Profile 的 YAML 详细描述了解每个领域的说明。

$ oc get profile.compliance -n openshift-compliance
NAME                       AGE   VERSION
ocp4-cis                   69s   1.5.0
ocp4-cis-1-4               70s   1.4.0
ocp4-cis-1-5               70s   1.5.0
ocp4-cis-node              69s   1.5.0
ocp4-cis-node-1-4          69s   1.4.0
ocp4-cis-node-1-5          69s   1.5.0
ocp4-e8                    68s
ocp4-high                  68s   Revision 4
ocp4-high-node             68s   Revision 4
ocp4-high-node-rev-4       68s   Revision 4
ocp4-high-rev-4            68s   Revision 4
ocp4-moderate              68s   Revision 4
ocp4-moderate-node         68s   Revision 4
ocp4-moderate-node-rev-4   68s   Revision 4
ocp4-moderate-rev-4        68s   Revision 4
ocp4-nerc-cip              68s
ocp4-nerc-cip-node         68s
ocp4-pci-dss               68s   3.2.1
ocp4-pci-dss-3-2           68s   3.2.1
ocp4-pci-dss-node          68s   3.2.1
ocp4-pci-dss-node-3-2      68s   3.2.1
ocp4-stig                  68s   V1R1
ocp4-stig-node             68s   V1R1
ocp4-stig-node-v1r1        68s   V1R1
ocp4-stig-v1r1             68s   V1R1
rhcos4-e8                  63s
rhcos4-high                63s   Revision 4
rhcos4-high-rev-4          63s   Revision 4
rhcos4-moderate            63s   Revision 4
rhcos4-moderate-rev-4      63s   Revision 4
rhcos4-nerc-cip            62s
rhcos4-stig                62s   V1R1
rhcos4-stig-v1r1           62s   V1R1

详细的合规 Profile 说明可参见 https://docs.openshift.com/container-platform/4.15/security/compliance_operator/co-scans/compliance-operator-supported-profiles.html

通过 labe 只查看和“rhcos4”相关的Profile。

$ oc get profile.compliance -l compliance.openshift.io/profile-bundle=rhcos4 -n openshift-compliance
NAME                    AGE    VERSION
rhcos4-e8               5m6s
rhcos4-high             5m6s   Revision 4
rhcos4-high-rev-4       5m6s   Revision 4
rhcos4-moderate         5m6s   Revision 4
rhcos4-moderate-rev-4   5m6s   Revision 4
rhcos4-nerc-cip         5m5s
rhcos4-stig             5m5s   V1R1
rhcos4-stig-v1r1        5m5s   V1R1

查看名为“rhcos4-e8”的Profile中包含的rule。

$ oc get profile.compliance rhcos4-e8 -n openshift-compliance -o json | jq .rules
[
  "rhcos4-accounts-no-uid-except-zero",
  "rhcos4-audit-rules-dac-modification-chmod",
  "rhcos4-audit-rules-dac-modification-chown",
  "rhcos4-audit-rules-execution-chcon",
  "rhcos4-audit-rules-execution-restorecon",
  "rhcos4-audit-rules-execution-semanage",
  "rhcos4-audit-rules-execution-setfiles",
  "rhcos4-audit-rules-execution-setsebool",
  "rhcos4-audit-rules-execution-seunshare",
  "rhcos4-audit-rules-kernel-module-loading-delete",
  "rhcos4-audit-rules-kernel-module-loading-finit",
  "rhcos4-audit-rules-kernel-module-loading-init",
  "rhcos4-audit-rules-login-events",
  "rhcos4-audit-rules-login-events-faillock",
  "rhcos4-audit-rules-login-events-lastlog",
  "rhcos4-audit-rules-login-events-tallylog",
  "rhcos4-audit-rules-networkconfig-modification",
  "rhcos4-audit-rules-sysadmin-actions",
  "rhcos4-audit-rules-time-adjtimex",
  "rhcos4-audit-rules-time-clock-settime",
  "rhcos4-audit-rules-time-settimeofday",
  "rhcos4-audit-rules-time-stime",
  "rhcos4-audit-rules-time-watch-localtime",
  "rhcos4-audit-rules-usergroup-modification",
  "rhcos4-auditd-data-retention-flush",
  "rhcos4-auditd-freq",
  "rhcos4-auditd-local-events",
  "rhcos4-auditd-log-format",
  "rhcos4-auditd-name-format",
  "rhcos4-auditd-write-logs",
  "rhcos4-configure-crypto-policy",
  "rhcos4-configure-ssh-crypto-policy",
  "rhcos4-no-empty-passwords",
  "rhcos4-selinux-policytype",
  "rhcos4-selinux-state",
  "rhcos4-service-auditd-enabled",
  "rhcos4-sshd-disable-empty-passwords",
  "rhcos4-sshd-disable-gssapi-auth",
  "rhcos4-sshd-disable-rhosts",
  "rhcos4-sshd-disable-root-login",
  "rhcos4-sshd-disable-user-known-hosts",
  "rhcos4-sshd-do-not-permit-user-env",
  "rhcos4-sshd-enable-strictmodes",
  "rhcos4-sshd-print-last-log",
  "rhcos4-sshd-set-loglevel-info",
  "rhcos4-sysctl-kernel-dmesg-restrict",
  "rhcos4-sysctl-kernel-kptr-restrict",
  "rhcos4-sysctl-kernel-randomize-va-space",
  "rhcos4-sysctl-kernel-unprivileged-bpf-disabled",
  "rhcos4-sysctl-kernel-yama-ptrace-scope",
  "rhcos4-sysctl-net-core-bpf-jit-harden"
]

还可查看Compliance Operator包含的所有rule。

$ oc get rule.compliance -n openshift-compliance
NAME                                                                                AGE
ocp4-accounts-restrict-service-account-tokens                                       11m
ocp4-accounts-unique-service-account                                                11m
ocp4-api-server-admission-control-plugin-alwaysadmit                                11m
ocp4-api-server-admission-control-plugin-alwayspullimages                           11m
ocp4-api-server-admission-control-plugin-namespacelifecycle                         11m
ocp4-api-server-admission-control-plugin-noderestriction                            11m
ocp4-api-server-admission-control-plugin-scc                                        11m
ocp4-api-server-admission-control-plugin-securitycontextdeny                        11m
ocp4-api-server-admission-control-plugin-serviceaccount                             11m
ocp4-api-server-anonymous-auth                                                      11m
ocp4-api-server-api-priority-flowschema-catch-all                                   11m
ocp4-api-server-api-priority-gate-enabled                                           11m
ocp4-api-server-api-priority-v1alpha1-flowschema-catch-all                          11m
ocp4-api-server-audit-log-maxbackup                                                 11m
ocp4-api-server-audit-log-maxsize                                                   11m
ocp4-api-server-audit-log-path                                                      11m
ocp4-api-server-auth-mode-no-aa                                                     11m
ocp4-api-server-auth-mode-node                                                      11m
ocp4-api-server-auth-mode-rbac                                                      11m
ocp4-api-server-basic-auth                                                          11m
ocp4-api-server-bind-address                                                        11m
ocp4-api-server-client-ca                                                           11m
。。。。

查看一个rule的详细信息，其中 description 部分是该 rule 是检查合规内容的说明。

$ oc get rule.compliance rhcos4-accounts-no-uid-except-zero -n openshift-compliance -oyaml
apiVersion: compliance.openshift.io/v1alpha1
checkType: Node
description: |-
  If any account other than root has a UID of 0, this misconfiguration should be investigated and the accounts other than root should be removed or have their UID changed.

  If the account is associated with system commands or applications the UID should be changed to one greater than "0" but less than "1000." Otherwise assign a UID greater than "1000" that has not already been assigned.
id: xccdf_org.ssgproject.content_rule_accounts_no_uid_except_zero
instructions: |-
  To list all password file entries for accounts with UID 0, run the
  following command:
  $ awk -F: '($3 == \"0\") {print}' /etc/passwd
  This should print only one line, for the user root.

  If there is a finding, change the UID of the failing (non-root) user. If
  the account is associated with the system commands or applications the UID
  should be changed to one greater than 0 but less than
  1000. Otherwise assign a UID of greater than 1000 that
  has not already been assigned.
kind: Rule
metadata:
  annotations:
    compliance.openshift.io/image-digest: pb-rhcos4pc5jz
    compliance.openshift.io/rule: accounts-no-uid-except-zero
    control.compliance.openshift.io/NERC-CIP: CIP-003-8 R5.1.1;CIP-003-8 R5.3;CIP-004-6
      R2.2.3;CIP-004-6 R2.3;CIP-007-3 R5.1;CIP-007-3 R5.1.2;CIP-007-3 R5.2;CIP-007-3
      R5.3.1;CIP-007-3 R5.3.2;CIP-007-3 R5.3.3
    control.compliance.openshift.io/NIST-800-53: IA-2;AC-6(5);IA-4(b)
    policies.open-cluster-management.io/controls: CIP-003-8 R5.1.1,CIP-003-8 R5.3,CIP-004-6
      R2.2.3,CIP-004-6 R2.3,CIP-007-3 R5.1,CIP-007-3 R5.1.2,CIP-007-3 R5.2,CIP-007-3
      R5.3.1,CIP-007-3 R5.3.2,CIP-007-3 R5.3.3,IA-2,AC-6(5),IA-4(b)
    policies.open-cluster-management.io/standards: NERC-CIP,NIST-800-53
  creationTimestamp: "2022-06-16T15:27:27Z"
  generation: 1
  labels:
    compliance.openshift.io/profile-bundle: rhcos4
  name: rhcos4-accounts-no-uid-except-zero
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ProfileBundle
    name: rhcos4
    uid: 9571f5e3-4577-4267-8b9a-b14ae40858e2
  resourceVersion: "478301"
  uid: 164d4595-144a-4651-8fe8-ace1d7fdc0c7
rationale: An account has root authority if it has a UID of 0. Multiple accounts with
  a UID of 0 afford more opportunity for potential intruders to guess a password for
  a privileged account. Proper configuration of sudo is recommended to afford multiple
  system administrators access to root privileges in an accountable manner.
severity: high
title: Verify Only Root Has UID 0

查看当前已有的 ScanSetting，它定义了合规扫描如何执行。

$ oc get ScanSetting -n openshift-compliance
NAME                 AGE
default              4h18m
default-auto-apply   4h18m

查看两者的 YAML，主要是以下 2 行的区别，其中 default 只做扫描，而 default-auto-apply 会自动修复发现的问题。

autoUpdateRemediations: true
autoApplyRemediations: true

合规扫描

配置定时合规扫描

创建一个新的 ScanSetting，每 30 分钟对 worker 节点进行一次扫描。为每个节点分配 1G 空间保留扫描结果，循环保留最后 5 个扫描结果。

$ cat << EOF | oc apply -f -
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSetting
metadata:
  name: periodic-setting
  namespace: openshift-compliance
schedule: "0/30 0 * * *"
rawResultStorage:
    size: "1Gi"
    rotation: 5
roles:
  - worker
EOF

创建一个新的 ScanSettingBinding，扫描使用 “rhcos4-e8” 和“ ocp4-e8” 两个Profile，并且使用名为 “periodic-setting” 的 ScanSetting 配置。

$ cat << EOF | oc apply -f -
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: periodic-e8
  namespace: openshift-compliance
profiles:
  # Node checks
  - name: rhcos4-e8
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
  # Platform checks
  - name: ocp4-e8
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: periodic-setting
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
EOF

通过 compliancesuite 查看合规扫描整体进度。由于需要持续运行一段时间，直到 PHASE 显示 DONE 即运行完成。

$ oc get compliancesuite -n openshift-compliance -w
NAME          PHASE     	RESULT
periodic-e8   PENDING   	NOT-AVAILABLE
periodic-e8   LAUNCHING   	NOT-AVAILABLE
periodic-e8   RUNNING   	NOT-AVAILABLE
periodic-e8   RUNNING   	NOT-AVAILABLE
periodic-e8   RUNNING   	NOT-AVAILABLE
periodic-e8   AGGREGATING   NOT-AVAILABLE
periodic-e8   DONE          NON-COMPLIANT

另外还可通过 compliancescan 查看 compliancesuite 中每个合规扫描的执行情况。

$ oc get compliancescan -n openshift-compliance
NAME               PHASE       RESULT
ocp4-e8            LAUNCHING   NOT-AVAILABLE
rhcos4-e8-worker   LAUNCHING   NOT-AVAILABLE

查看名为 “ocp4-e8” 的合规扫描的结果 compliancecheckresult，其中状态为 PASS 代表通过扫描 Rule、FAIL 代表没有通过扫描 Rule、MANUAL 代表可以手动检查。

$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=ocp4-e8
NAME                                               STATUS   SEVERITY
ocp4-e8-api-server-encryption-provider-cipher      FAIL     medium
ocp4-e8-api-server-tls-cipher-suites               PASS     medium
ocp4-e8-ocp-allowed-registries                     FAIL     medium
ocp4-e8-ocp-allowed-registries-for-import          FAIL     medium
ocp4-e8-ocp-idp-no-htpasswd                        FAIL     medium
ocp4-e8-rbac-limit-cluster-admin                   MANUAL   medium
ocp4-e8-rbac-pod-creation-access                   MANUAL   medium
ocp4-e8-rbac-wildcard-use                          MANUAL   medium
ocp4-e8-scc-limit-container-allowed-capabilities   PASS     medium
ocp4-e8-scc-limit-privilege-escalation             MANUAL   medium
ocp4-e8-scc-limit-privileged-containers            MANUAL   medium
ocp4-e8-scc-limit-root-containers                  MANUAL   medium

查看名为 “ocp4-e8” 的合规扫描结果 compliancecheckresults，且 “check-status=FAIL” 或 “check-severity=medium”。

$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=ocp4-e8,compliance.openshift.io/check-status=FAIL
NAME                                            STATUS   SEVERITY
ocp4-e8-api-server-encryption-provider-cipher   FAIL     medium
ocp4-e8-ocp-allowed-registries                  FAIL     medium
ocp4-e8-ocp-allowed-registries-for-import       FAIL     medium
ocp4-e8-ocp-idp-no-htpasswd                     FAIL     medium
 
$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=ocp4-e8,compliance.openshift.io/check-severity=medium
NAME                                               STATUS   SEVERITY
ocp4-e8-api-server-encryption-provider-cipher      FAIL     medium
ocp4-e8-api-server-tls-cipher-suites               PASS     medium
ocp4-e8-ocp-allowed-registries                     FAIL     medium
ocp4-e8-ocp-allowed-registries-for-import          FAIL     medium
ocp4-e8-ocp-idp-no-htpasswd                        FAIL     medium
ocp4-e8-rbac-limit-cluster-admin                   MANUAL   medium
ocp4-e8-rbac-pod-creation-access                   MANUAL   medium
ocp4-e8-rbac-wildcard-use                          MANUAL   medium
ocp4-e8-scc-limit-container-allowed-capabilities   PASS     medium
ocp4-e8-scc-limit-privilege-escalation             MANUAL   medium
ocp4-e8-scc-limit-privileged-containers            MANUAL   medium
ocp4-e8-scc-limit-root-containers                  MANUAL   medium

查看名为 “rhcos4-e8-master” 和 “rhcos4-e8-worker” 的合规扫描结果 compliancecheckresults。

$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=rhcos4-e8-worker

统计不同状态的扫描结果数量。

$ echo -n PASS: && oc get compliancecheckresult -n openshift-compliance | grep PASS | wc -l && \
echo -n FAIL: && oc get compliancecheckresult -n openshift-compliance | grep FAIL | wc -l && \
echo -n INFO: && oc get compliancecheckresult -n openshift-compliance | grep INFO | wc -l && \
echo -n MANUAL: && oc get compliancecheckresult -n openshift-compliance | grep MANUAL | wc -l && \
echo -n NOT-APPLICABLE: && oc get compliancecheckresult -n openshift-compliance | grep NOT-APPLICABLE | wc -l
 
PASS:13
FAIL:43
INFO:0
MANUAL:6
NOT-APPLICABLE:0

执行命令查看 complianceremediation，可以看到返回结果的 STATE 为 NotApplied 状态，代表 Operator 不会自动修复违规项目。

$ oc get complianceremediation -n openshift-compliance
NAME                                                        STATE
ocp4-e8-api-server-encryption-provider-cipher               NotApplied
rhcos4-e8-worker-audit-rules-dac-modification-chmod         NotApplied
rhcos4-e8-worker-audit-rules-dac-modification-chown         NotApplied
rhcos4-e8-worker-audit-rules-execution-chcon                NotApplied
rhcos4-e8-worker-audit-rules-execution-restorecon           NotApplied
rhcos4-e8-worker-audit-rules-execution-semanage             NotApplied
rhcos4-e8-worker-audit-rules-execution-setfiles             NotApplied
rhcos4-e8-worker-audit-rules-execution-setsebool            NotApplied
rhcos4-e8-worker-audit-rules-execution-seunshare            NotApplied
rhcos4-e8-worker-audit-rules-kernel-module-loading-delete   NotApplied
...

手动执行扫描

通过为已有 compliancescan 对象配置 annotate 可以手动执行扫描。

$ oc annotate compliancescans/rhcos4-e8-worker compliance.openshift.io/rescan=

获取扫描结果文件

可以用以下2种方法任意一种获得打包的扫描结果：

通过辅助 Pod 获取扫描结果

此种方法需要OpenShift中配置有storageclass。

查看所有compliancescan的扫描结果保存的PV名称。

$ oc get compliancescans -n openshift-compliance -o json | jq '.items[].status.resultsStorage'
{
  "name": "ocp4-e8",
  "namespace": "openshift-compliance"
}
{
  "name": "rhcos4-e8-worker",
  "namespace": "openshift-compliance"
}

验证在OpenShift中已经有 “rhcos4-e8-master” 的 PV 了，它是用来保存 rhcos4-e8-master 扫描结果的 PV。

$ oc get pvc -n openshift-compliance
NAME                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
ocp4-e8                Bound    pvc-7e363d19-69d7-4bde-939a-74212e9cf9d3   1Gi        RWO            gp2            33m
rhcos4-e8-worker       Bound    pvc-00a3ed90-d608-436b-84e0-40ea435937d8   1Gi        RWO            gp2            34m

创建一个pod，也将其挂载到上一步看到的名为“rhcos4-e8-worker”的PV。

$ cat << EOF | oc apply -f -
apiVersion: "v1"
kind: Pod
metadata:
  name: pv-extract-rhcos4-e8-worker
  namespace: openshift-compliance
spec:
  containers:
    - name: pv-extract-pod
      image: registry.access.redhat.com/ubi8/ubi
      command: ["sleep", "3000"]
      volumeMounts:
        - mountPath: "/worker-scan-results"
          name: worker-scan-vol
  volumes:
    - name: worker-scan-vol
      persistentVolumeClaim:
        claimName: rhcos4-e8-worker
EOF

确认名为“pv-extract”的 pod 已经运行起来。

$ oc get pod pv-extract -n openshift-compliance
NAME                          READY   STATUS    RESTARTS   AGE
pv-extract-rhcos4-e8-worker   1/1     Running   0          34s

将名为“pv-extract”的pod的合规扫描结果复制到本地“extract_results_dir”目录中。

$ oc -n openshift-compliance cp pv-extract-rhcos4-e8-worker:/worker-scan-results ./extract_results_dir
tar: Removing leading `/' from member names

查看本地目录中已经包含扫描结果的打包文件。

$ cd extract_results_dir/0/ && ll 
total 588
-rw-r--r--. 1 lab-user users 601164 Jun 18 03:18 rhcos4-e8-worker-ip-10-0-149-198.us-east-2.compute.internal-pod.xml.bzip2

通过 oc-compliance 命令插件获取扫描结果

此种方法需要有 registry.redhat.io 账号用来下载 oc-compliance 命令插件，该插件可用来简化使用 oc 命令对合规结果进行操作。

创建使用的目录，并安装必要的工具和 openscap-scanner。

$ mkdir -p ~/.local/bin
$ podman login -u <USER> -p <PASSWORD> registry.redhat.io
$ podman run --rm --entrypoint /bin/cat registry.redhat.io/compliance/oc-compliance-rhel8 /usr/bin/oc-compliance > ~/.local/bin/oc-compliance
Trying to pull registry.redhat.io/compliance/oc-compliance-rhel8...
Getting image source signatures
Copying blob f0ae454850a7 done
Copying blob 053724d29990 done
Copying blob ecbbd3f38c20 done
Copying config df71917de3 done
Writing manifest to image destination
Storing signatures
 
$ chmod +x ~/.local/bin/oc-compliance

使用 oc-compliance 获取和名为 periodic-e8 的 scansettingbinding 相关的扫描结果，保存到指定目录。

$ mkdir /tmp/periodic-e8
$ oc-compliance fetch-raw scansettingbinding periodic-e8 -n openshift-compliance -o /tmp/periodic-e8
Fetching results for periodic-e8 scans: rhcos4-e8-worker, ocp4-e8
Fetching raw compliance results for scan 'rhcos4-e8-worker'.....
The raw compliance results are avaliable in the following directory: /tmp/periodic-e8/rhcos4-e8-worker
Fetching raw compliance results for scan 'ocp4-e8'......
The raw compliance results are avaliable in the following directory: /tmp/periodic-e8/ocp4-e8

确认获取的扫描结果中的内容。

$ tree /tmp/periodic-e8/
/tmp/periodic-e8/
├── ocp4-e8
│   └── ocp4-e8-api-checks-pod.xml.bzip2
└── rhcos4-e8-worker
    ├── rhcos4-e8-worker-ip-10-0-149-198.us-east-2.compute.internal-pod.xml.bzip2

查看扫描结果

查看所有标签有 “rhcos4-e8-worker” 的合规扫描结果，确认其中有 “rhcos4-e8-worker-selinux-state”。

$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=rhcos4-e8-worker | grep rhcos4-e8-worker-selinux-state
rhcos4-e8-worker-selinux-state                              PASS     medium

查看扫描结果详细情况

$ oc-compliance view-result rhcos4-e8-worker-selinux-state -n openshift-compliance
+----------------------+--------------------------------+
|         KEY          |             VALUE              |
+----------------------+--------------------------------+
| Title                | Ensure SELinux State is        |
|                      | Enforcing                      |
+----------------------+--------------------------------+
| Status               | PASS                           |
+----------------------+--------------------------------+
| Severity             | medium                         |
+----------------------+--------------------------------+
| Description          | The SELinux state should be    |
|                      | set to enforcing at system     |
|                      | boot time. In the file         |
|                      | /etc/selinux/config , add or   |
|                      | correct the following line to  |
|                      | configure the system to boot   |
|                      | into enforcing mode:           |
|                      |                                |
|                      | SELINUX= enforcing             |
+----------------------+--------------------------------+
| Rationale            | Setting the SELinux state to   |
|                      | enforcing ensures SELinux is   |
|                      | able to confine potentially    |
|                      | compromised processes to the   |
|                      | security policy, which is      |
|                      | designed to prevent them from  |
|                      | causing damage to the system   |
|                      | or further elevating their     |
|                      | privileges.                    |
+----------------------+--------------------------------+
| Instructions         | Ensure that Red Hat Enterprise |
|                      | Linux CoreOS 4 verifies        |
|                      | correct operation of security  |
|                      | functions.                     |
|                      |                                |
|                      | Check if "SELinux" is active   |
|                      | and in "" mode with the        |
|                      | following command:             |
|                      |                                |
|                      | $ sudo getenforce              |
+----------------------+--------------------------------+
| NERC-CIP Controls    | CIP-003-8 R5.1.1, CIP-003-8    |
|                      | R5.2, CIP-003-8 R5.3,          |
|                      | CIP-004-6 R2.2.3, CIP-004-6    |
|                      | R2.3, CIP-004-6 R3.3,          |
|                      | CIP-007-3 R5.1, CIP-007-3      |
|                      | R5.1.2, CIP-007-3 R5.2,        |
|                      | CIP-007-3 R5.3.1, CIP-007-3    |
|                      | R5.3.2, CIP-007-3 R5.3.3,      |
|                      | CIP-007-3 R6.5                 |
+----------------------+--------------------------------+
| NIST-800-53 Controls | AC-3, AC-3(3)(a), AU-9,        |
|                      | SC-7(21)                       |
+----------------------+--------------------------------+
| Available Fix        | No                             |
+----------------------+--------------------------------+
| Result Object Name   | rhcos4-e8-worker-selinux-state |
+----------------------+--------------------------------+
| Rule Object Name     | rhcos4-selinux-state           |
+----------------------+--------------------------------+
| Remediation Created  | No                             |
+----------------------+--------------------------------+

查看扫描报告

安装 openscap-scanner 和其他工具

$ sudo yum install openscap-scanner bzip2 -y

将结果从XML格式文件解压出来，然后转换成 HTML 格式。

$ mkdir /tmp/periodic-e8/ocp4-e8 -p 
$ bunzip2 -c /tmp/periodic-e8/ocp4-e8/ocp4-e8-api-checks-pod.xml.bzip2  > /tmp/periodic-e8/ocp4-e8/ocp4-e8-api-checks-pod.xml
$ oscap xccdf generate report /tmp/periodic-e8/ocp4-e8/ocp4-e8-api-checks-pod.xml > /tmp/periodic-e8/ocp4-e8/report.html

然后可以打开 HTML 格式的扫描报告。

合规修复

批量修复

在 OpenShift 控制台的 Compliance Operator 中修改名为 default-auto-apply 的 ScanSetting 对象，只为 roles 保留 master。
说明：因为本文使用的集群只有一个节点，既是 master 也是 worker。如果 master 和 worker 分开的集群可以忽略此步。

...省略
roles:
  - master
...省略

执行命令，基于 rhcos4-moderate profile 和 default-auto-apply scansetting 创建一个新的 ScanSettingBinding。其中 default-auto-apply 代表 Compliance Operator 会自动根据生成的 ComplianceRemediation 修复违规项目；而 rhcos4-moderate 是对 OpenShift 集群的 RHCOS 进行的合规扫描。

$ cat << EOF | oc apply -f -
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: rhcos4-moderate
  namespace: openshift-compliance
profiles:
  # Node checks
  - name: rhcos4-moderate
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default-auto-apply
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
EOF

查看当前 machineconfig 对象的情况，确认只有 00、01、99 开头的配置项目。

$ oc get machineconfig 
00-master                                                                                              d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
00-worker                                                                                              d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
01-master-container-runtime                                                                            d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
01-master-kubelet                                                                                      d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
01-worker-container-runtime                                                                            d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
01-worker-kubelet 
99-master-generated-crio-seccomp-use-default                                                                                                      3.2.0             3h30m
99-master-generated-registries                                                                         d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
99-master-ssh                                                                                                                                     3.2.0             3h33m
99-worker-generated-crio-seccomp-use-default                                                                                                      3.2.0             3h30m
99-worker-generated-registries                                                                         d493389f76a7c2d26ac074f67ee1f0e73329315b   3.2.0             3h30m
99-worker-ssh

在一个新窗口运行以下命令，持续观察集群节点状态。注意：本文使用的集群只有一个节点，既是 master 也是 worker。

$ oc get node -w
NAME                                         STATUS   ROLES           AGE     VERSION
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h29m   v1.23.5+8471591
...

查看合规扫描过程，直到 DONE 。

$ oc get compliancescan -n openshift-compliance -w | grep rhcos4-moderate
NAME                     PHASE   RESULT
rhcos4-moderate-master   DONE    NON-COMPLIANT

执行命令查看 complianceremediation ，可以看到返回结果的 STATE 大都是 Applied 状态，代表 Operator 自动修复这些项目。

$ oc get complianceremediation -n openshift-compliance

再次查看 machineconfig 对象，确认增加了很多名称以 75-rhcos4-moderate 开头的 machineconfig ，这些即为修复 RHCOS 安全配置的 machineconfig 配置。

$ oc get machineconfig

查看集群节点状态，确认会从 Ready 变为 SchedulingDisabled 状态。最后 node 将会被 reboot 从而使新 machineconfig 配置生效。

$ oc get node -w
NAME                                         STATUS   ROLES           AGE    VERSION
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h4m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready    master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready,SchedulingDisabled   master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready,SchedulingDisabled   master,worker   3h5m   v1.23.5+8471591
ip-10-0-206-198.us-east-2.compute.internal   Ready,SchedulingDisabled   master,worker   3h6m   v1.23.5+8471591

集群节点恢复 Ready 后违规修复即完成。
执行命令统计上一次扫描结果为 PASS 的项目数量。

$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=rhcos4-moderate-master | grep PASS | wc -l

执行命令，重新执行一次名为 rhcos4-moderate-master 的 compliancescan 扫描。

$ oc -n openshift-compliance annotate compliancescans/rhcos4-moderate-master compliance.openshift.io/rescan=

等扫描完成并且node完成重启后，再次统计扫描结果为 PASS 的项目数量，确认数量已经增加，这说明修复成功。

$ oc get compliancecheckresult -n openshift-compliance -l compliance.openshift.io/scan-name=rhcos4-moderate-master | grep PASS | wc -l

逐一修复

执行以下命令，创建基于 rhcos4-high profile 和 default scansetting 创建一个新的 ScanSettingBinding。其中 default 代表 Compliance Operator 不会自动根据生成的 ComplianceRemediation 进行违规修复，需要人工实施 ComplianceRemediation 方能生效修复过程。

$ cat << EOF | oc apply -f -
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: rhcos4-high
  namespace: openshift-compliance
profiles:
  # Node checks
  - name: rhcos4-high
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
EOF

执行以下命令，将名为 rhcos4-high-master-service-sshd-disabled 违规修复的 spec.apply 设置为 true，实现单独修其对应的违规项目。

$ oc -n openshift-compliance patch complianceremediations/rhcos4-high-master-service-sshd-disabled --patch '{"spec":{"apply":true}}' --type=merge

执行以下命令，确认 Compliance Operator 自动根据 complianceremediations/rhcos4-high-master-service-sshd-disabled 生成 machineconfig 对象。

$ oc get mc | grep rhcos4-high-master-service-sshd-disabled
75-rhcos4-high-master-service-sshd-disabled                                                  3.1.0             53m

执行命令，重新执行一次名为 rhcos4-high-master 的 compliancescan 扫描。

$ oc -n openshift-compliance annotate compliancescans/rhcos4-high-master compliance.openshift.io/rescan=

执行以下命令，确认名为 rhcos4-high-master-service-sshd-disabled 的扫描结果已经是 PASS 状态了。

$ oc get compliancecheckresult rhcos4-high-master-service-sshd-disabled -n openshift-compliance
NAME                                       STATUS   SEVERITY
rhcos4-high-master-service-sshd-disabled   PASS     unknown

撤销合规修复

先确认一下当前名为 rhcos4-high-master-service-sshd-disabled 的合规扫描结果是 PASS 状态。

$ oc get compliancecheckresult rhcos4-high-master-service-sshd-disabled -n openshift-compliance
NAME                                       STATUS   SEVERITY
rhcos4-high-master-service-sshd-disabled   PASS     unknown

执行命令，修改名为 rhcos4-high-master-service-sshd-disabled 的 complianceremediation 配置，将 spec.apply 设为 false。

$ oc -n openshift-compliance patch complianceremediations/rhcos4-high-master-service-sshd-disabled --patch '{"spec":{"apply":false}}' --type=merge

确认以前由 Compliance Operator 生成的名为 rhcos4-high-master-service-sshd-disabled 的 machineconfig 也没有了。

$ oc get mc | grep rhcos4-high-master-service-sshd-disabled

执行命令，重新执行一次名为 rhcos4-high-master 的 compliancescan 扫描。

$ oc -n openshift-compliance annotate compliancescans/rhcos4-high-master compliance.openshift.io/rescan=

确认此时名为 rhcos4-high-master-service-sshd-disabled 的合规扫描结果回退成 FAIL 状态。

$ oc get compliancecheckresult rhcos4-high-master-service-sshd-disabled -n openshift-compliance -w
NAME                                       STATUS   SEVERITY
rhcos4-high-master-service-sshd-disabled   FAIL     unknown

演示视频

参考

https://github.com/openshift/compliance-operator/tree/master/doc/tutorials
https://blog.stderr.at/compliance/2021/07/compliance-operator/
https://shanna-chan.blog/2021/11/17/getting-started-on-openshift-compliance-operator/
https://github.com/openshift/oc-compliance
https://myopenshiftblog.com/openshift-compliance-operator/
https://github.com/pittar/ocp4-compliance-pbmm