[k8s]kubespray(ansible)自动化安装k8s集群

最新推荐文章于 2023-05-04 11:40:31 发布
最新推荐文章于 2023-05-04 11:40:31 发布
阅读量490 收藏
点赞数
文章标签： 运维 开发工具 python

kubespray(ansible)自动化安装k8s集群

https://github.com/kubernetes-incubator/kubespray

https://kubernetes.io/docs/setup/pick-right-solution/

kubespray本质是一堆ansible的role文件,通过这种方式,即ansible方式可以自动化的安装高可用k8s集群,目前支持1.9.
806469-20171228121556066-642557996.png

安装完成后,k8s所有组件都是通过hyperkube的容器化来运行的.

  • 最佳安装centos7

  • 安装docker(kubespray默认给docker添加的启动参数)

/usr/bin/dockerd
--insecure-registry=10.233.0.0/18
--graph=/var/lib/docker
--log-opt max-size=50m
--log-opt max-file=5
--iptables=false
--dns 10.233.0.3
--dns 114.114.114.114
--dns-search default.svc.cluster.local
--dns-search svc.cluster.local
--dns-opt ndots:2
--dns-opt timeout:2
--dns-opt attempts:2

- 规划3(n1  n2 n3)主2从(n4 n5)

- hosts

192.168.2.11 n1.ma.com n1
192.168.2.12 n2.ma.com n2
192.168.2.13 n3.ma.com n3
192.168.2.14 n4.ma.com n4
192.168.2.15 n5.ma.com n5
192.168.2.16 n6.ma.com n6


- 1.9的ansible-playbook里涉及到这些镜像
gcr.io/google_containers/cluster-proportional-autoscaler-amd64:1.1.1
gcr.io/google_containers/pause-amd64:3.0
gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7
gcr.io/google_containers/elasticsearch:v2.4.1
gcr.io/google_containers/fluentd-elasticsearch:1.22
gcr.io/google_containers/kibana:v4.6.1
gcr.io/kubernetes-helm/tiller:v2.7.2
gcr.io/google_containers/kubernetes-dashboard-amd64:v1.0.1
gcr.io/google_containers/kubernetes-dashboard-init-amd64:v1.7.1
quay.io/l23network/k8s-netchecker-agent:v1.0
quay.io/l23network/k8s-netchecker-server:v1.0
quay.io/coreos/etcd:v3.2.4
quay.io/coreos/flannel:v0.9.1
quay.io/coreos/flannel-cni:v0.3.0
quay.io/calico/ctl:v1.6.1
quay.io/calico/node:v2.6.2
quay.io/calico/cni:v1.11.0
quay.io/calico/kube-controllers:v1.0.0
quay.io/calico/routereflector:v0.4.0
quay.io/coreos/hyperkube:v1.9.0_coreos.0
quay.io/ant31/kargo:master
quay.io/external_storage/local-volume-provisioner-bootstrap:v1.0.0
quay.io/external_storage/local-volume-provisioner:v1.0.0

- 1.9镜像dockerhub版
lanny/gcr.io_google_containers_cluster-proportional-autoscaler-amd64:1.1.1
lanny/gcr.io_google_containers_pause-amd64:3.0
lanny/gcr.io_google_containers_k8s-dns-kube-dns-amd64:1.14.7
lanny/gcr.io_google_containers_k8s-dns-dnsmasq-nanny-amd64:1.14.7
lanny/gcr.io_google_containers_k8s-dns-sidecar-amd64:1.14.7
lanny/gcr.io_google_containers_elasticsearch:v2.4.1
lanny/gcr.io_google_containers_fluentd-elasticsearch:1.22
lanny/gcr.io_google_containers_kibana:v4.6.1
lanny/gcr.io_kubernetes-helm_tiller:v2.7.2
lanny/gcr.io_google_containers_kubernetes-dashboard-init-amd64:v1.0.1
lanny/gcr.io_google_containers_kubernetes-dashboard-amd64:v1.7.1
lanny/quay.io_l23network_k8s-netchecker-agent:v1.0
lanny/quay.io_l23network_k8s-netchecker-server:v1.0
lanny/quay.io_coreos_etcd:v3.2.4
lanny/quay.io_coreos_flannel:v0.9.1
lanny/quay.io_coreos_flannel-cni:v0.3.0
lanny/quay.io_calico_ctl:v1.6.1
lanny/quay.io_calico_node:v2.6.2
lanny/quay.io_calico_cni:v1.11.0
lanny/quay.io_calico_kube-controllers:v1.0.0
lanny/quay.io_calico_routereflector:v0.4.0
lanny/quay.io_coreos_hyperkube:v1.9.0_coreos.0
lanny/quay.io_ant31_kargo:master
lanny/quay.io_external_storage_local-volume-provisioner-bootstrap:v1.0.0
lanny/quay.io_external_storage_local-volume-provisioner:v1.0.0



- 配置文件
kubespray/inventory/group_vars/k8s-cluster.yml  为控制一些基础信息的配置文件。
修改为flannel

kubespray/inventory/group_vars/all.yml           控制一些需要详细配置的信息 
修改为centos

- 替换flannel的vxlan为host-gw
roles/network_plugin/flannel/defaults/main.yml
可以通过grep -r 'vxlan' .  这种方式来找到

- 修改kube_api_pwd
vi roles/kubespray-defaults/defaults/main.yaml
kube_api_pwd: xxxx

- 证书时间
kubespray/roles/kubernetes/secrets/files/make-ssl.sh



- 修改image为个人仓库地址
可以通过grep -r 'gcr.io' .  这种方式来找到
sed -i 's#gcr\.io\/google_containers\/#lanny/gcr\.io_google_containers_#g' roles/download/defaults/main.yml
sed -i 's#gcr\.io\/google_containers\/#lanny/gcr\.io_google_containers_#g' roles/dnsmasq/templates/dnsmasq-autoscaler.yml.j2
sed -i 's#gcr\.io\/google_containers\/#lanny/gcr\.io_google_containers_#g' roles/kubernetes-apps/ansible/defaults/main.yml
sed -i 's#gcr\.io\/kubernetes-helm\/#lanny/gcr\.io_kubernetes-helm_#g' roles/download/defaults/main.yml

sed -i 's#quay\.io\/coreos\/#lanny/quay\.io_coreos_#g' roles/download/defaults/main.yml
sed -i 's#quay\.io\/calico\/#lanny/quay\.io_calico_#g' roles/download/defaults/main.yml
sed -i 's#quay\.io\/l23network\/#lanny/quay\.io_l23network_#g' roles/download/defaults/main.yml
sed -i 's#quay\.io\/l23network\/#lanny/quay\.io_l23network_#g' docs/netcheck.md
sed -i 's#quay\.io\/external_storage\/#lanny/quay\.io_external_storage_#g' roles/kubernetes-apps/local_volume_provisioner/defaults/main.yml
sed -i 's#quay\.io\/ant31\/kargo#lanny/quay\.io_ant31_kargo_#g' .gitlab-ci.yml





- 安装完后查看镜像(用到的镜像(calico))
nginx:1.13
quay.io/coreos/hyperkube:v1.9.0_coreos.0
gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7
gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
quay.io/calico/node:v2.6.2
gcr.io/google_containers/kubernetes-dashboard-init-amd64:v1.0.1
quay.io/calico/cniv:1.11.0
gcr.io/google_containers/kubernetes-dashboard-amd64:v1.7.1
quay.io/calico/ctlv:1.6.1
quay.io/calico/routereflectorv:0.4.0
quay.io/coreos/etcdv:3.2.4
gcr.io/google_containers/cluster-proportional-autoscaler-amd64:1.1.1
gcr.io/google_containers/pause-amd64:3.0


- kubespray生成配置所需的环境(python3 ansible) Ansible v2.4 (or newer) Jinja 2.9 (or newer) 
yum install python34 python34-pip python-pip  python-netaddr -y
cd
mkdir .pip
cd .pip
cat > pip.conf <<EOF
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/

[install]
trusted-host=mirrors.aliyun.com
EOF

yum install gcc libffi-devel python-devel openssl-devel -y
pip install  Jinja2-2.10-py2.py3-none-any.whl # https://pypi.python.org/pypi/Jinja2
pip install cryptography
pip install ansible



- 克隆
git clone https://github.com/kubernetes-incubator/kubespray.git


- 修改配置

1. 使docker能pull gcr的镜像
vim inventory/group_vars/all.yml
2 bootstrap_os: centos
95 http_proxy: "http://192.168.1.88:1080/"

2. 如果vm内存<=1G,如果>=3G,则无需修改
vim roles/kubernetes/preinstall/tasks/verify-settings.yml 
52 - name: Stop if memory is too small for masters
53   assert:
54     that: ansible_memtotal_mb <= 1500
55   ignore_errors: "{{ ignore_assert_errors }}"
56   when: inventory_hostname in groups['kube-master']
57 
58 - name: Stop if memory is too small for nodes
59   assert:
60     that: ansible_memtotal_mb <= 1024              
61   ignore_errors: "{{ ignore_assert_errors }}"


3. 修改swap,
vim roles/download/tasks/download_container.yml
 75 - name: Stop if swap enabled
 76   assert:
 77     that: ansible_swaptotal_mb == 0         
 78   when: kubelet_fail_swap_on|default(false)

所有机器执行: 关闭swap
swapoff -a
[root@n1 kubespray]# free -m
              total        used        free      shared  buff/cache   available
Mem:           2796         297        1861           8         637        2206
Swap:             0           0           0  #这栏为0,表示关闭

- 生成 kubespray配置,开始ansible安装k8s之旅(非常用时间,大到1h,小到20min)
cd kubespray
IPS=(192.168.2.11 192.168.2.12 192.168.2.13 192.168.2.14 192.168.2.15)
CONFIG_FILE=inventory/inventory.cfg python3 contrib/inventory_builder/inventory.py ${IPS[@]}
ansible-playbook -i inventory/inventory.cfg cluster.yml -b -v --private-key=~/.ssh/id_rsa


- 修改好的配置
git clone https://github.com/lannyMa/kubespray
run.sh 有启动命令


- 我的inventory.cfg, 我不想让node和master混在一起手动改了下
[root@n1 kubespray]# cat inventory/inventory.cfg 
[all]
node1    ansible_host=192.168.2.11 ip=192.168.2.11
node2    ansible_host=192.168.2.12 ip=192.168.2.12
node3    ansible_host=192.168.2.13 ip=192.168.2.13
node4    ansible_host=192.168.2.14 ip=192.168.2.14
node5    ansible_host=192.168.2.15 ip=192.168.2.15

[kube-master]
node1    
node2    
node3

[kube-node]      
node4    
node5    

[etcd]
node1    
node2    
node3    

[k8s-cluster:children]
kube-node    
kube-master      

[calico-rr]

遇到的问题wait for the apiserver to be running

ansible-playbook -i inventory/inventory.ini cluster.yml -b -v --private-key=~/.ssh/id_rsa
...
<!-- We recommend using snippets services like https://gist.github.com/ etc. -->
RUNNING HANDLER [kubernetes/master : Master | wait for the apiserver to be running] ***
Thursday 23 March 2017  10:46:16 +0800 (0:00:00.468)       0:08:32.094 ******** 
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (10 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (10 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (9 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (9 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (8 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (8 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (7 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (7 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (6 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (6 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (5 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (5 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (4 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (4 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (3 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (3 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (2 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (2 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (1 retries left).
FAILED - RETRYING: HANDLER: kubernetes/master : Master | wait for the apiserver to be running (1 retries left).
fatal: [node1]: FAILED! => {"attempts": 10, "changed": false, "content": "", "failed": true, "msg": "Status code was not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://localhost:8080/healthz"}
fatal: [node2]: FAILED! => {"attempts": 10, "changed": false, "content": "", "failed": true, "msg": "Status code was not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://localhost:8080/healthz"}
        to retry, use: --limit @/home/dev_dean/kargo/cluster.retry

解决: 所有节点关闭swap

swapoff -a

806469-20171228000459597-535895199.png

806469-20171228000727816-1913513533.png

快捷命令

alias kk='kubectl get pod --all-namespaces -o wide --show-labels'
alias ks='kubectl get svc --all-namespaces -o wide'
alias kss='kubectl get svc --all-namespaces -o wide --show-labels'
alias kd='kubectl get deploy --all-namespaces -o wide'
alias wk='watch kubectl get pod --all-namespaces -o wide --show-labels'
alias kv='kubectl get pv -o wide'
alias kvc='kubectl get pvc -o wide --all-namespaces --show-labels'
alias kbb='kubectl run -it --rm --restart=Never busybox --image=busybox sh'
alias kbbc='kubectl run -it --rm --restart=Never curl --image=appropriate/curl sh'
alias kd='kubectl get deployment --all-namespaces --show-labels'
alias kcm='kubectl get cm --all-namespaces -o wide'
alias kin='kubectl get ingress --all-namespaces -o wide'

kubespray的默认启动参数

ps -ef|egrep  "apiserver|controller-manager|scheduler"


/hyperkube apiserver \
--advertise-address=192.168.2.11 \
--etcd-servers=https://192.168.2.11:2379,https://192.168.2.12:2379,https://192.168.2.13:2379 \
--etcd-quorum-read=true \
--etcd-cafile=/etc/ssl/etcd/ssl/ca.pem \
--etcd-certfile=/etc/ssl/etcd/ssl/node-node1.pem \
--etcd-keyfile=/etc/ssl/etcd/ssl/node-node1-key.pem \
--insecure-bind-address=127.0.0.1 \
--bind-address=0.0.0.0 \
--apiserver-count=3 \
--admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ValidatingAdmissionWebhook,ResourceQuota \
--service-cluster-ip-range=10.233.0.0/18 \
--service-node-port-range=30000-32767 \
--client-ca-file=/etc/kubernetes/ssl/ca.pem \
--profiling=false \
--repair-malformed-updates=false \
--kubelet-client-certificate=/etc/kubernetes/ssl/node-node1.pem \
--kubelet-client-key=/etc/kubernetes/ssl/node-node1-key.pem \
--service-account-lookup=true \
--tls-cert-file=/etc/kubernetes/ssl/apiserver.pem \
--tls-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem \
--proxy-client-cert-file=/etc/kubernetes/ssl/apiserver.pem \
--proxy-client-key-file=/etc/kubernetes/ssl/apiserver-key.pem \
--service-account-key-file=/etc/kubernetes/ssl/apiserver-key.pem \
--secure-port=6443 \
--insecure-port=8080 
--storage-backend=etcd3 \
--runtime-config=admissionregistration.k8s.io/v1alpha1 --v=2 \
--allow-privileged=true \
--anonymous-auth=False \
--authorization-mode=Node,RBAC \
--feature-gates=Initializers=False \
PersistentLocalVolumes=False


/hyperkube controller-manager \
--kubeconfig=/etc/kubernetes/kube-controller-manager-kubeconfig.yaml \
--leader-elect=true \
--service-account-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem \
--root-ca-file=/etc/kubernetes/ssl/ca.pem \
--cluster-signing-cert-file=/etc/kubernetes/ssl/ca.pem \
--cluster-signing-key-file=/etc/kubernetes/ssl/ca-key.pem \
--enable-hostpath-provisioner=false \
--node-monitor-grace-period=40s \
--node-monitor-period=5s \
--pod-eviction-timeout=5m0s \
--profiling=false \
--terminated-pod-gc-threshold=12500 \
--v=2 \
--use-service-account-credentials=true \
--feature-gates=Initializers=False \
PersistentLocalVolumes=False


/hyperkube scheduler \
--leader-elect=true \
--kubeconfig=/etc/kubernetes/kube-scheduler-kubeconfig.yaml \
--profiling=false --v=2 \
--feature-gates=Initializers=False \
PersistentLocalVolumes=False


/usr/local/bin/kubelet \
--logtostderr=true --v=2 \
--address=192.168.2.14 \
--node-ip=192.168.2.14 \
--hostname-override=node4 \
--allow-privileged=true \
--pod-manifest-path=/etc/kubernetes/manifests \
--cadvisor-port=0 \
--pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.0 \
--node-status-update-frequency=10s \
--docker-disable-shared-pid=True \
--client-ca-file=/etc/kubernetes/ssl/ca.pem \
--tls-cert-file=/etc/kubernetes/ssl/node-node4.pem \
--tls-private-key-file=/etc/kubernetes/ssl/node-node4-key.pem \
--anonymous-auth=false \
--cgroup-driver=cgroupfs \
--cgroups-per-qos=True \
--fail-swap-on=True \
--enforce-node-allocatable= \
--cluster-dns=10.233.0.3 \
--cluster-domain=cluster.local \
--resolv-conf=/etc/resolv.conf \
--kubeconfig=/etc/kubernetes/node-kubeconfig.yaml \
--require-kubeconfig \
--kube-reserved cpu=100m,memory=256M \
--node-labels=node-role.kubernetes.io/node=true \
--feature-gates=Initializers=False,PersistentLocalVolumes=False \
--network-plugin=cni --cni-conf-dir=/etc/cni/net.d \
--cni-bin-dir=/opt/cni/bin


ps -ef|grep kube-porxy
hyperkube proxy --v=2 \
    --kubeconfig=/etc/kubernetes/kube-proxy-kubeconfig.yaml\
    --bind-address=192.168.2.14 \
    --cluster-cidr=10.233.64.0/18 \
    --proxy-mode=iptables

我把所需要的镜像v1.9推送到dockerhub了

lanny/gcr.io_google_containers_cluster-proportional-autoscaler-amd64:1.1.1
lanny/gcr.io_google_containers_pause-amd64:3.0
lanny/gcr.io_google_containers_k8s-dns-kube-dns-amd64:1.14.7
lanny/gcr.io_google_containers_k8s-dns-dnsmasq-nanny-amd64:1.14.7
lanny/gcr.io_google_containers_k8s-dns-sidecar-amd64:1.14.7
lanny/gcr.io_google_containers_elasticsearch:v2.4.1
lanny/gcr.io_google_containers_fluentd-elasticsearch:1.22
lanny/gcr.io_google_containers_kibana:v4.6.1
lanny/gcr.io_kubernetes-helm_tiller:v2.7.2
lanny/gcr.io_google_containers_kubernetes-dashboard-init-amd64:v1.0.1
lanny/gcr.io_google_containers_kubernetes-dashboard-amd64:v1.7.1
lanny/quay.io_l23network_k8s-netchecker-agent:v1.0
lanny/quay.io_l23network_k8s-netchecker-server:v1.0
lanny/quay.io_coreos_etcd:v3.2.4
lanny/quay.io_coreos_flannel:v0.9.1
lanny/quay.io_coreos_flannel-cni:v0.3.0
lanny/quay.io_calico_ctl:v1.6.1
lanny/quay.io_calico_node:v2.6.2
lanny/quay.io_calico_cni:v1.11.0
lanny/quay.io_calico_kube-controllers:v1.0.0
lanny/quay.io_calico_routereflector:v0.4.0
lanny/quay.io_coreos_hyperkube:v1.9.0_coreos.0
lanny/quay.io_ant31_kargo:master
lanny/quay.io_external_storage_local-volume-provisioner-bootstrap:v1.0.0
lanny/quay.io_external_storage_local-volume-provisioner:v1.0.0




gcr.io/google_containers/cluster-proportional-autoscaler-amd64:1.1.1
gcr.io/google_containers/pause-amd64:3.0
gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7
gcr.io/google_containers/elasticsearch:v2.4.1
gcr.io/google_containers/fluentd-elasticsearch:1.22
gcr.io/google_containers/kibana:v4.6.1
gcr.io/kubernetes-helm/tiller:v2.7.2
gcr.io/google_containers/kubernetes-dashboard-amd64:v1.0.1
gcr.io/google_containers/kubernetes-dashboard-init-amd64:v1.7.1
quay.io/l23network/k8s-netchecker-agent:v1.0
quay.io/l23network/k8s-netchecker-server:v1.0
quay.io/coreos/etcd:v3.2.4
quay.io/coreos/flannel:v0.9.1
quay.io/coreos/flannel-cni:v0.3.0
quay.io/calico/ctl:v1.6.1
quay.io/calico/node:v2.6.2
quay.io/calico/cni:v1.11.0
quay.io/calico/kube-controllers:v1.0.0
quay.io/calico/routereflector:v0.4.0
quay.io/coreos/hyperkube:v1.9.0_coreos.0
quay.io/ant31/kargo:master
quay.io/external_storage/local-volume-provisioner-bootstrap:v1.0.0
quay.io/external_storage/local-volume-provisioner:v1.0.0


sed -i 's#gcr\.io\/google_containers\/#lanny/gcr\.io_google_containers_#g' roles/download/defaults/main.yml
sed -i 's#gcr\.io\/google_containers\/#lanny/gcr\.io_google_containers_#g' roles/dnsmasq/templates/dnsmasq-autoscaler.yml.j2
sed -i 's#gcr\.io\/google_containers\/#lanny/gcr\.io_google_containers_#g' roles/kubernetes-apps/ansible/defaults/main.yml
sed -i 's#gcr\.io\/kubernetes-helm\/#lanny/gcr\.io_kubernetes-helm_#g' roles/download/defaults/main.yml

sed -i 's#quay\.io\/coreos\/#lanny/quay\.io_coreos_#g' roles/download/defaults/main.yml
sed -i 's#quay\.io\/calico\/#lanny/quay\.io_calico_#g' roles/download/defaults/main.yml
sed -i 's#quay\.io\/l23network\/#lanny/quay\.io_l23network_#g' roles/download/defaults/main.yml
sed -i 's#quay\.io\/l23network\/#lanny/quay\.io_l23network_#g' docs/netcheck.md
sed -i 's#quay\.io\/external_storage\/#lanny/quay\.io_external_storage_#g' roles/kubernetes-apps/local_volume_provisioner/defaults/main.yml
sed -i 's#quay\.io\/ant31\/kargo#lanny/quay\.io_ant31_kargo_#g' .gitlab-ci.yml

参考: https://jicki.me/2017/12/08/kubernetes-kubespray-1.8.4/

构建etcd容器集群

/usr/bin/docker run
    --restart=on-failure:5 
    --env-file=/etc/etcd.env
    --net=host
    -v /etc/ssl/certs:/etc/ssl/certs:ro
    -v /etc/ssl/etcd/ssl:/etc/ssl/etcd/ssl:ro
    -v /var/lib/etcd:/var/lib/etcd:rw
    --memory=512M
    --oom-kill-disable
    --blkio-weight=1000
    --name=etcd1
    lanny/quay.io_coreos_etcd:v3.2.4
    /usr/local/bin/etcd

[root@node1 ~]# cat /etc/etcd.env 
ETCD_DATA_DIR=/var/lib/etcd
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.2.11:2379
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.2.11:2380
ETCD_INITIAL_CLUSTER_STATE=existing
ETCD_METRICS=basic
ETCD_LISTEN_CLIENT_URLS=https://192.168.2.11:2379,https://127.0.0.1:2379
ETCD_ELECTION_TIMEOUT=5000
ETCD_HEARTBEAT_INTERVAL=250
ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd
ETCD_LISTEN_PEER_URLS=https://192.168.2.11:2380
ETCD_NAME=etcd1
ETCD_PROXY=off
ETCD_INITIAL_CLUSTER=etcd1=https://192.168.2.11:2380,etcd2=https://192.168.2.12:2380,etcd3=https://192.168.2.13:2380
ETCD_AUTO_COMPACTION_RETENTION=8

# TLS settings
ETCD_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCD_CERT_FILE=/etc/ssl/etcd/ssl/member-node1.pem
ETCD_KEY_FILE=/etc/ssl/etcd/ssl/member-node1-key.pem
ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCD_PEER_CERT_FILE=/etc/ssl/etcd/ssl/member-node1.pem
ETCD_PEER_KEY_FILE=/etc/ssl/etcd/ssl/member-node1-key.pem
ETCD_PEER_CLIENT_CERT_AUTH=true



[root@node1 ssl]# docker exec b1159a1c6209 env|grep -i etcd
ETCD_DATA_DIR=/var/lib/etcd
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.2.11:2379
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.2.11:2380
ETCD_INITIAL_CLUSTER_STATE=existing
ETCD_METRICS=basic
ETCD_LISTEN_CLIENT_URLS=https://192.168.2.11:2379,https://127.0.0.1:2379
ETCD_ELECTION_TIMEOUT=5000
ETCD_HEARTBEAT_INTERVAL=250
ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd
ETCD_LISTEN_PEER_URLS=https://192.168.2.11:2380
ETCD_NAME=etcd1
ETCD_PROXY=off
ETCD_INITIAL_CLUSTER=etcd1=https://192.168.2.11:2380,etcd2=https://192.168.2.12:2380,etcd3=https://192.168.2.13:2380
ETCD_AUTO_COMPACTION_RETENTION=8
ETCD_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCD_CERT_FILE=/etc/ssl/etcd/ssl/member-node1.pem
ETCD_KEY_FILE=/etc/ssl/etcd/ssl/member-node1-key.pem
ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCD_PEER_CERT_FILE=/etc/ssl/etcd/ssl/member-node1.pem
ETCD_PEER_KEY_FILE=/etc/ssl/etcd/ssl/member-node1-key.pem
ETCD_PEER_CLIENT_CERT_AUTH=true


[root@node1 ssl]# tree /etc/ssl/etcd/ssl
/etc/ssl/etcd/ssl
├── admin-node1-key.pem
├── admin-node1.pem
├── admin-node2-key.pem
├── admin-node2.pem
├── admin-node3-key.pem
├── admin-node3.pem
├── ca-key.pem
├── ca.pem
├── member-node1-key.pem
├── member-node1.pem
├── member-node2-key.pem
├── member-node2.pem
├── member-node3-key.pem
├── member-node3.pem
├── node-node1-key.pem
├── node-node1.pem
├── node-node2-key.pem
├── node-node2.pem
├── node-node3-key.pem
├── node-node3.pem
├── node-node4-key.pem
├── node-node4.pem
├── node-node5-key.pem
└── node-node5.pem

[root@n2 ~]# tree /etc/ssl/etcd/ssl
/etc/ssl/etcd/ssl
├── admin-node2-key.pem
├── admin-node2.pem
├── ca-key.pem
├── ca.pem
├── member-node2-key.pem
├── member-node2.pem
├── node-node1-key.pem
├── node-node1.pem
├── node-node2-key.pem
├── node-node2.pem
├── node-node3-key.pem
├── node-node3.pem
├── node-node4-key.pem
├── node-node4.pem
├── node-node5-key.pem
└── node-node5.pem

admin-node1.crt
CN = etcd-admin-node1

DNS Name=localhost
DNS Name=node1
DNS Name=node2
DNS Name=node3
IP Address=192.168.2.11
IP Address=192.168.2.11
IP Address=192.168.2.12
IP Address=192.168.2.12
IP Address=192.168.2.13
IP Address=192.168.2.13
IP Address=127.0.0.1

member-node1.crt
CN = etcd-member-node1

DNS Name=localhost
DNS Name=node1
DNS Name=node2
DNS Name=node3
IP Address=192.168.2.11
IP Address=192.168.2.11
IP Address=192.168.2.12
IP Address=192.168.2.12
IP Address=192.168.2.13
IP Address=192.168.2.13
IP Address=127.0.0.1

node-node1.crt
CN = etcd-node-node1

DNS Name=localhost
DNS Name=node1
DNS Name=node2
DNS Name=node3
IP Address=192.168.2.11
IP Address=192.168.2.11
IP Address=192.168.2.12
IP Address=192.168.2.12
IP Address=192.168.2.13
IP Address=192.168.2.13
IP Address=127.0.0.1


[root@node1 bin]# tree 
.
├── etcd
├── etcdctl
├── etcd-scripts
│   └── make-ssl-etcd.sh
├── kubectl
├── kubelet
└── kubernetes-scripts
    ├── kube-gen-token.sh
    └── make-ssl.sh
weixin_34378922
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
【Java】企业微信机器人自动发送群消息
分数没二舅血压高
05-19 4015
1.先在群里添加机器人，然后获取机器人的webhook地址： 假设webhook是：https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=693a91f6-7xxx-4bc4-97a0-0ec2sifa5aaa 2.有多种方式发送群消息，可以采用curl，也可以采用发送https请求的方式，我这里采用okhttp发送http请求，pom如下： <!-- https://mvnrepository.com/artifact/com.squareup
Ansible自动化部署K8S集群（包含Ansible，Playbook的使用）
02-17
Ansible自动化部署Kubernetes，超级简单，几分钟就可以部署完成，有部署过程及源代码，Ansible的简单使用以及playbook的使用例子，有完整代码
python request 报错ConnectionRefusedError: [Errno 111] Connection refused
zhouzy00001的博客
12-23 6570
调试接口请求的时候，报连接错误， 后来发现，是端口错了。试了下，ip搞错也会这样报错的 『『需自动化项目源码\交流学习添加v：ct-xiaozhou 免费提供的。行业变迁，多个好友多点可能性。非机构』』 详细报错如下： Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 170, in _new_conn (sel...
企业微信如何简单实现定时发送文件到群：企业微信群机器人操作（Java代码实现）
坐等夕阳落time的博客
12-24 8854
前言 不知道小伙伴们的公司组织架构通勤用的啥软件，我公司用的企业微信。然后业务销售部那边需要每天统计销售数据报表然后发在群里（我是开发，我不配在群里）。知道这个背景以后，产品给我们的需求是：直接统计数据按照业务那边的报表模板直接生成销售报表，然后定时每天晚上9点发送到各个门店的相关群（企业微信群）里。 实现 第一步：肯定要先去看企业微信的开发文档啊 1、开发文档 群机器人配置说明 - 企业微信API 2、稍微说一下我看文档后的理解 群机器人作用： 企业微信群机器人怎么发送消息： ..
企业微信 WebHook 实现故障文本告警
m0_56693018的博客
05-04 1146
我们再开发企业微信机器人可以直接通过 JavaScript 语言编写脚本，因为得到了 Zabbix 的原生支持，告警脚本通用性强且更加灵活。如下图，你也可以选择启动机器人的时间段，比如我要 7x24 小时盯着网络设备，这里直接 007 就好了。注意，这个 WebHook 地址非常重要，请不要泄露，否则他人可以通过这个地址给你捣乱。在“Test media type”对话框中，可以直接点击“测试”，也可以自定义通知的主题和内容，如图。这样，当有设备触发器被触发产生告警时，企业微信机器人就会给你发消息了。
Shell 企业微信机器人发送告警
山炮运维
06-24 3030
#!/bin/bash wx(){ #将下面的webhook地址替换成你的企业微信机器人地址，$1为告警消息 $2为@人的手机号 $2可以为空 cat > $0.msg << EOF curl 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=d25d7339-9b15-4b17-80ae-2f546140c349' \ -H 'Content-Type: application/json' \ -d ' {
ansible自动化安装k8s集群
02-16
ansible自动化安装k8s集群1.18.8+,测试有效，5分钟搞定，自己以及IDC机房都用得这个。 脚本和安装文档都有。
K8s集群搭建ansible部署脚本
04-21
我个人给公司开发的使用ansible部署k8s的脚本，支持vagrant调用ansbile，和直接ansible执行两种方式。k8s二进制组件使用最新的1.23.5 部署以下模块内容包括： preinstall 安装前准备，主机环境初始化，二进制文件...
基于Ubutntu系统使用Ansible一键部署K8S1.24.17容器版集群自动化工具
10-12
2、支持 单机、一主多从、三主多从等 三种部署架构和 六种部署模式的容器版K8S离线集群， 证书有效期为99年。 3、支持 一键部署、分步部署、集群备份（etcd）、集群恢复（etcd）、集群移除 等功能。 4、支持 worker...
Ansible脚本搭建K8s集群.pptx
12-25
利用ansible搭建k8s的具体操作文档
python wechat --- 企业微信机器人API
zly717216的博客
09-12 5713
python企业微信机器人API库，支持群机器人相关API调用，包括发送文本消息、图片、markdown、文件、模板消息卡片、模板图片卡片。基于webhook开发，调用简单，文档详细易读...
企业微信机器人 api 使用
Dreamy_LIN的博客
10-17 1948
企业微信机器人 api 使用
shell脚本触发企业微信群机器人
1853
05-19 2708
shell脚本触发企业微信群机器人 1、创建企业微信 略 2、企业微信添加机器人 3、复制hook地址 https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=08149ccf-3668-4fb9-bb06-asdfasdfasd 4、编辑shell脚本 #!/bin/bash WEBHOOK_URL="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=08149ccf-3668-4fb9-bb
git lfs原理和使用
热门推荐
HowieMen的博客
09-27 1万+
如果我们用git管理的项目中出现了一些大文件, 同时若其数量比较多, 而且更新又比较频繁, 那么当首次clone该项目时, 就会不可避免地将这些大文件的当前版本和历史所有版本的文件都下载下来, 虽然你很可能用不到这些历史文件, 但是却不得不为它们所占用的额外空间买单. **git lfs**(large file storage)就很好地解决了这一问题. 通过将大文件替换为小指针的方式, 当你真正需要到这些大文件的时候, 它才会从本地或者远端的lfs缓存中下载这些大文件.
【容器】Kargo装k8 1.6报这个错误
毛台
04-01 1719
Kargo装k8 1.6报这个错误 RUNNING HANDLER [kubernetes/master : Master | wait for the apiserver to be running] *** Saturday 01 April 2017 11:44:32 +0800 (0:00:00.627) 0:11:21.278 ******** FAILED - R
Ansible自动化部署高可用K8S集群
qq_42409495的博客
12-08 2264
1.1 Ansible介绍 Ansible 是一种IT自动化工具。它可以配置系统，部署软件以及协调更高级的IT任务，例如持续部署，滚动更新。Ansible 适用于管理企业IT基础设施，从具有少数主机的小规模到数千个实例的企业环境。Ansible 也是一种简单的自动化语言，可以完美地描述IT应用程序基础结构。 具备以下三个特点： 简单：减少学习成本 强大：协调应用程序生命周期 无代理：可预测，可靠和安全 使用文档：https://releases.ansible.com/ansible/ 安装 A
google浏览器打不开网页解决办法，返回ERR_TUNNEL_CONNECTION_FAILED“
dafanpai的博客
11-23 8434
google浏览器打不开网页解决办法，而且显示：“网页可能暂时无法连接，或者它已永久性地移动到了新网址，返回ERR_TUNNEL_CONNECTION_FAILED” 打开cmd: 输入： ipconfig /flushdns 亲测刷新一下dns配置就可以访问了，如果还不行在一次输入下面的指令后，重启电脑就可以了，但多数情况win10应该都不用重启 nbtstat –r netsh int ip reset netsh winsock reset ...
阿里开源事件告警组件kube-eventer，让我们花5分钟一起实战体验
技术宅，专注云原生、Go、Linux、Java等技术分享，原创文章、效率工具、实战解决方案！关注我，了解互联网动态，学 IT 不迷路
12-31 1979
背景 监控是保障系统稳定性的重要组成部分，在Kubernetes开源生态中，资源类的监控工具与组件监控百花齐放。 cAdvisor：kubelet内置的的cAdvisor，监控容器资源，如容器cpu、内存； Kube-state-metrics：kube-state-metrics通过监听 API Server 生成有关资源对象的状态指标，主要关注元数据，比如 * Deployment、Pod、...
ansible的安装与使用+kubespray部署k8s
唐僧爱程序
08-05 334
-安装epel库 # yum -y install epel-release 安装pip，python3自带pip3 # yum -y install python-pip # 升级pip pip install --upgrade pip # 检查版本 pip --version pip 19.0.3 from /usr/lib/python2.7/site-packages/pip (python 2.7) 通过pip安装ansible等必须工具 # pip install ansible
k8s集群ansible 节点互信、
最新发布
12-08
以下是k8s集群ansible节点互信的步骤： 1.在ansible控制节点上生成ssh密钥对，如果已经有密钥对可以跳过此步骤。 ```shell ssh-keygen -t rsa -b 2048 ``` 2.将公钥复制到所有节点上，这里以复制到node1为例。 ```...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值