openshift 入门

sanzhixiaohou

于 2024-09-02 16:27:43 发布

阅读量582

点赞数 4

文章标签： openshift

本文链接：https://blog.csdn.net/sanzhixiaohou/article/details/141820062

版权

1，登录 ocp

1.1 portal 上查看 api token

portal address ：https://address/dashboards

Your API token is
sha256~xxxx
Log in with this token
oc login --token=sha256~xxxx --server=https://api.address:6443
Use this token directly against the API
curl -H "Authorization: Bearer sha256~xxxx" "https://api.address:6443/apis/user.openshift.io/v1/users/~"

1.2 bastion node 上直接登录：

export KUBECONFIG=/ocp_install/auth/kubeconfig
oc get node
oc get csr
oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs --no-run-if-empty oc adm certificate approve

2， portal 登录默认账户 kubeadmin

安装时候显示 kubeadmin 密码：
NFO Login to the console with user: "kubeadmin", and password: "xxxxxxx"
如果密码忘了也没关系，可以查看文件 /ocp_install/auth/kubeadmin-password 来获得密码。[root@bastion ~]# find / -name kubeadmin-password
/ocp_install/auth/kubeadmin-password
[root@bastion ~]# cd /ocp_install/auth/
[root@bastion auth]# ls
kubeadmin-password kubeconfig
[root@bastion auth]# cat kubeadmin-password
xxxxxxx

3， ocp 常用命令

oc get nodes //获取集群所有节点
oc describe node node-name //查看对应节点详细信息，可以看到运行在该节点下的pod
oc get pods -n namespace-name //查看对应namespace下pod
oc describe pod pod-name -n namespace-name //查看pod详细信息
oc get limitrange -n namespace-name //获取对应namespace的limitrange配置文件
oc describe limitrange limitrange.config -n namespace-name //查看配置文件详情
oc edit limitrange limitrange.config -n namespace-name //修改limitrange配置
oc project project-name //切换到project
oc adm policy add-scc-to-user anyuid -z default //为该project开启anyuid，
可以使用root权限，一般是安装/运行某些软件时需要
oc describe clusterrole.rbac //查看集群管理员角色及权限
oc describe clusterrolebinding.rbac //查看用户组及绑定的角色
oc adm policy add-cluster-role-to-user cluster-admin username //添加username为cluster-admin
oc get routes --all-namespace //查看所有namespace的route
oc logs -f pod-name //查看pod log
docker ps -a|grep pod-name //查看pod对应containerID
docker exec -it containerID /bin/sh //登录到container

查看所有namespace 用-A
[root@bastion auth]# oc get namespace
NAME                                               STATUS   AGE
0001                                           Active   430d
default                                            Active   447d

[root@bastion auth]# oc get ingress -A
No resources found
[root@bastion auth]# oc get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
0001 example

4，例子：

4.1 部署

新建一个project 0001zyq

部署两个deployment：

1，切换到developer 模式，点Add，选择 Container images。 Image 选 nginx 部署nginx，选python部署python，部署时候都遇到错误，解决方法如下：
2:
部署 nginx 错误：
[root@bastion ~]# oc logs nginx-eample-75b65c8d98-xcvvf
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/10/27 06:28:18 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2023/10/27 06:28:18 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
[root@bastion ~]#
原因：
openshift里面的scc（security context constrains），安全上下文约束。也就是授权。例如访问project的权限，访问api的权限。需要把权限授权给用户。
Using project "0001zyq".
解决方法：

1，Grant a service account Access to the Privileged SCC
1.1 create a service account , for example to create service account 0001zyqsvacct in project 0001zyq:
1.2 add the service account to the privileged SCC.
[root@bastion ~]# oc create serviceaccount 0001zyqsvacct -n 0001zyq
serviceaccount/0001zyqsvacct created
[root@bastion ~]# oadm policy add-scc-to-user privileged system:serviceaccount:0001zyq:0001zyqsvacct
-bash: oadm: command not found
[root@bastion ~]# oc adm policy add-scc-to-user privileged system:serviceaccount:0001zyq:0001zyqsvacct
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:privileged added: "0001zyqsvacct"
2,Enable Images to run with user in the docker file
To relax the security in your cluster so that images are not forced to run as a pre-allocated UID, without granting everyone access to the prevailed SCC

2.1 Grant all authenticated users access to the anyuid SCC:
[root@bastion ~]# oc adm policy add-scc-to-group anyuid system:authenticated
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "system:authenticated"
[root@bastion ~]#
this allows images to run as the root UID if no USER is specified in the Dockerfile.

按照上述方法delete nginx 重建，状态正常。系统默认的scc策略是 openshift.io/scc:restricted, 上述方法改成 openshift.io/scc:anyuid

另一个solved方法：
oc edit scc anyuid

Add
Users:
-system:serviceaccount:default:default

问题3 ：
Nignx 启动失败，因为scc为，需要更改为anyuid：
openshift.io/scc: restricted-v2

问题解决：
[root@bastion ~]# oc get pods | grep python
python-test-5c5855b97d-2mzrg    1/1     Running            0                 3m38s
查看1号进程：
# ps -ef
UID          PID    PPID C STIME TTY          TIME CMD
root           1       0 0 06:37 ?        00:00:00 tail -f /dev/null
root           7       0 0 06:42 pts/0    00:00:00 sh -i -c TERM=xterm sh

拓展：
pid=1的进程：1号进程，init进程。
Linux启动后，第一个被创建的用户态进程就是init进程，作用：
1，执行系统初始化脚本，创建一系列进程
2，在一个死循环中等待其子进程的退出事件，并调用waitid系统调用来完成“收尸”工作
Init进程不会被暂停也不会被删除
Pid为0 和1 的进程不会被kill，系统进行初始化时候吧这两个进程的所有信号屏蔽了，通过kill发送给0，1的信号被忽略，不会有任何作用。

tail -f /dev/null 毫无意义，只是一个无限运行又不使用任何CPU的命令。
如果没有这个命令，当shuf命令完成后，容器讲停止，有了这个容器将保持空闲状态，

问题：
浏览器无法访问部署好的nginx
查看route，nginx地址为：
https://nginx-eample-0001zyq.apps.qq412.cdl.ibm.com/
解决：
sudo vim /etc/hosts
添加：
9.123.108.20 nginx-eample-0001zyq.apps.qq412.cdl.ibm.com

问题：
python pod status CrashLoopBackOff 一直重启，没有logs:

[root@bastion ~]# oc get pods | grep python
python-test-78bf869494-65nkt    0/1     CrashLoopBackOff   1683 (4m18s ago)   5d23h
[root@bastion ~]# oc logs python-test-78bf869494-65nkt
[root@bastion ~]#
[root@bastion ~]# oc get event|grep python
116s        Warning   BackOff   pod/python-test-78bf869494-65nkt   Back-off restarting failed container
[root@bastion ~]#
[root@bastion ~]# oc describe pod python-test-78bf869494-65nkt
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Thu, 02 Nov 2023 14:30:14 +0800
      Finished:     Thu, 02 Nov 2023 14:30:14 +0800

Exit Code: 0 表示是正常退出
原因：没有pid=1的常驻进程，在deployment 的yaml配置：
command:[“/bin/bash”,”-ce”,”tail -f /dev/null”]
是image封装的问题，没有指定启动参数，需要使用者在使用iamge时候，自己添加启动命令和参数（command，args），如果不添加，容器内没有正确的执行命令，运行即退出

例如：
kind: Deployment
apiVersion: apps/v1
metadata:
annotations:
    alpha.image.policy.openshift.io/resolve-names: '*'
resourceVersion: '81903175'
name: python-test
spec:
replicas: 1
selector:
    matchLabels:
      app: python-test
template:
    metadata:
      creationTimestamp: null
      labels:
        app: python-test
        deployment: python-test
      annotations:
        openshift.io/generated-by: OpenShiftWebConsole
    spec:
      containers:
        - name: python-test
          image: >-
            image-registry.openshift-image-registry.svc:5000/0001zyq/python-test@sha256:833b16f07978d42d8005190eb2d00c74c63ad9c8aad12c83c9c45c7fc7a79903
          ports:
            - containerPort: 8080
              protocol: TCP
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: Always
          command: ["/bin/bash","-ce","tail -f /dev/null"]

save and reload。

5，环境增加用户：

为环境增加新用户：

You are logged in as a temporary administrative user. Update the cluster OAuth configuration to allow others to log in.
参考：https://zhuanlan.zhihu.com/p/148394928?utm_id=0

为环境增加新用户：
1, htpasswd -c -B -b users.htpasswd user password
[root@bastion auth]# htpasswd -c -B -b users.htpasswd zyqzeng zyqzeng
Adding password for user zyqzeng
[root@bastion auth]# ls
kubeadmin-password kubeconfig users.htpasswd

users.htpasswd是新生成的文件。
3,切换 admin 模式，在administration->Cluster setting-> Configuration(或者是Global Configuration)，点击OAuth，点击add ， htpasswd。上传 users.htpasswd
4,New identity provider added
点击 htpasswd 可以用zyqzeng 登录了。
可以editOauth删除已经配置的用户
这一步需要比较长的一段时间，需要等一会
新用户zyqzeng 登录后不能看到Cluster Setting

如果增加admin：
[root@bastion auth]# htpasswd -c -B -b users.htpasswd admin 12345
Adding password for user admin
[root@bastion auth]# ls
kubeadmin-password kubeconfig users.htpasswd
htpasswd：admin会有比普通用户更多的权限。能看到Cluster Setting，根据版本，有的能看到Oauth details，有的不能，执行下命令给admin授予cluster-admin 权限。

oc adm policy add-cluster-role-to-user cluster-admin admin
可以删除默认kubeadmin，不建议删除：
oc -n kube-system delete secrets kubeadmin

6，

OCP 环境中安装react
1，创建一个 node deployment， yaml中添加 command:[“/bin/bash”,”-ce”,”tail -f /dev/null”]
    spec:
      containers:
        - name: node-test1
          image: >-
            image-registry.openshift-image-registry.svc:5000/0001zyq/node-test1@sha256:b632a1ec41a0927d19a432c525641b9a4451251d0b0b63f1d764810a562ea4e1
          command:
            - /bin/bash
            - '-ce'
            - tail -f /dev/null
          ports:
            - containerPort: 8080
              protocol: TCP
2,pod 内部安装react，启动。查看podyaml中 podIP: 10.131.0.66

在pod内部和pod之间可以访问 http://10.131.0.66:3000
外界无法访问。
3，Create networking service

kind: Service
apiVersion: v1
metadata:
name: node-test1-nodeport
namespace: 0001zyq
labels:
    app: node-test1
spec:
ports:
    - protocol: TCP
      port: 80
      targetPort: 3000               #pod暴露端口
      nodePort: 31080                #外部访问端口
internalTrafficPolicy: Cluster
type: NodePort                     #指定service 类型，外部访问需要是NodePort
ipFamilyPolicy: SingleStack
sessionAffinity: None
selector:
    app: node-test1                  #关联的app名字

http://bastionip:31080/ 不能访问 react
任意一个非bastion node地址，例如 http://worknode:31080/ 可以访问react

7，两个pod之间通信：

两个pod之间的API通信：
这里用 python-test 和 python-test-api2为例子：
1，在 python-test 上：
Apt-get update
Apt-get install vim
pip install fastapi
pip install uvicorn
然后，创建一个Python文件，例如main.py，并编写以下代码：
from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: str = None):
    return {"item_id": item_id, "q": q}
这个代码定义了两个API接口：
    •    /：当你访问根URL时，它会返回一个包含Hello键和World值的JSON对象。
    •    /items/{item_id}: 当你访问这个URL并提供一个item_id和一个可选的查询参数q时，它会返回一个JSON对象。
运行这个API服务器：

uvicorn main:app --reload //默认8000端口
uvicorn main:app --host 0.0.0.0 --port 8080 8080duankou

现在，你可以通过访问http://127.0.0.1:8000/或http://127.0.0.1:8000/items/123?q=example来测试这两个接口。
curl http://127.0.0.1:8000/或http://127.0.0.1:8000/items/123?q=example

2, Creat deployment时候，同时有生成 service。
可以查看service里面的 name 和 port，
Port：对外暴露的 port
tergetport：api的port，需要和 pod yaml里的port 一致，

疑问：所有端口都配置成8080。可以访问别的pod API，都配置成8000为什么不醒
在 python-test 访问 python-test-api2。
# curl http://python-test-api2:8080/items/123?q=example
{"item_id":123,"qqqqqq":"example"}#
python-test-api2 上同时配置了route，可以在外面访问：
https://python-test-api2-0001zyq.apps.qq412.cdl.ibm.com/

https://python-test-api2-0001zyq.apps.qq412.cdl.ibm.com/items/123?q=example

8，ocp安装ingress

Ingress 配置安装下列方式：
https://blog.csdn.net/u010834071/article/details/139884443
Ingress 报错：
Error "failed calling webhook "validate.nginx.ingr

oc delete -A ValidatingWebhookConfiguration ingress-nginx-admission
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted

ingress 增加annotations，例如默认cookie成 same-site: Lax 修改 ingress yaml
annotations:
router.openshift.io/cookie-same-site: Lax

命令行添加Annotations：

添加Annotations
oc annotate route nginx-eample2 'openshift.io/samesite=Lax' -n 0001zyq

oc annotate route nginx-eample2 'router.openshift.io/cookie-same-site=Lax' -n 0001zyq
删除Annotations
oc annotate route <route_name> example.com/my-annotation-

oc annotate route nginx-eample2 'router.openshift.io/cookie-same-site=Lax'- -n 0001zyq

9，image

为了能操作内部的 image registry，需要给自己赋权，例如用admin登录后执行以下命令：
[root@bastion ~]# oc policy add-role-to-user registry-viewer $(oc whoami)
clusterrole.rbac.authorization.k8s.io/registry-viewer added: "admin"
[root@bastion ~]# oc policy add-role-to-user registry-editor $(oc whoami)
clusterrole.rbac.authorization.k8s.io/registry-editor added: "admin"
[root@bastion ~]#

如果不赋权后面会报错：
sh-4.4# podman push image-registry.openshift-image-registry.svc:5000/0001zyq/hello-openshift:latest
Getting image source signatures
Error: trying to reuse blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef at destination: checking whether a blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef exists in image-registry.openshift-image-registry.svc:5000/0001zyq/hello-openshift: unauthorized: authentication required

ingress映射内部的ip