目录
上一章讲了StatefulSet的相关知识,现在讲一个拓扑状态和存储状态结合的例子。
一、例子说明
1.1 功能说明
我这里的是起2个mysql绑定不同的两个持久卷,我使用的是是local,没在使用其它系统,比如ceph或GFS。
也可以看k8s官网的statefulSet的mysql结合的有状态应用例子
PS:我这里就不实现mysql主从了,因为mysql主从HA有更好的组件mysql-operator,可以实现比StatefulSet更复杂的功能。
operator是k8s的一个扩展插件,功能更强,可以看一下,懂go语言是最好的,不懂那就使用现成的,别人写好的吧。
因为locat不支持动态,所以要不要sc都没关系,k8s也没有现成的pv,所以需要自己建立一个,但有参照例子。
1.2 实现思路
主要难点就是StatefulSet生成的pod绑定的不同的pvc,这样才能达到不同的mysql运行在不同的目录中。
注解思路,可以安装2个 storageClassName相关的pv,只是名字不而已,接入方式使用 ReadWriteOnce只能绑定一次。
二、例子实现
2.1 准备工作(node)
#为了加快,先在节点上,把镜像下载下来,我这里只有节点1 vm821
docker pull mysql:8.0.21
#在指定的节点vm821和vm822上创建目录
mkdir -p /disk1/mysql8/data
mkdir -p /disk1/mysql8/data1
2.2 建立pv(master)
如果不懂得建立local pv,可以参照官网的local卷
#master节点操作:
mkdir -p /disk1/myk8s
cd /disk1/myk8s/
cat>mysql8-localpv.yaml<<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
name: data-mysql8-0
labels:
pv: data-mysql8-0
spec:
capacity:
storage: 30Gi
#默认是Filesystem,建议使用Block以得到更好的性能,但是发现改变为Block之后不能绑定
volumeMode: Filesystem
accessModes:
#只能有一个读写
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: data-mysql8
local:
path: /disk1/mysql8/data
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
#获取hostname变量值
- key: kubernetes.io/arch
operator: In
values:
- amd64
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: data-mysql8-1
labels:
pv: data-mysql8-1
spec:
capacity:
storage: 30Gi
volumeMode: Filesystem
accessModes:
#只能有一个读写
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: data-mysql8
local:
path: /disk1/mysql8/data1
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
#获取hostname变量值
- key: kubernetes.io/arch
operator: In
values:
- amd64
EOF
#创建pv卷
#cat mysq-localpv1.yaml
kubectl apply -f mysql8-localpv.yaml
注:
- 2个pv的storageClassName值是相同的
storageClassName: data-mysql8
- 如果不懂节点选择器可以使用 kubectl get node --show-labels 命令
从web管理中看如下图所示:运行正常
2.3 建立sc(master 可选)
可参照官网的建立本地sc
#local不支持自动化,可以建立sc是可选项
cat>mysql8-local-sv.yaml<<EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
#名字要与pv的名字一致
name: data-mysql8
#no-provisioner
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF
cat mysql8-local-sv.yaml
kubectl apply -f mysql8-local-sv.yaml
本地卷还不支持动态分配,然而还是需要创建 StorageClass 以延迟卷绑定,直到完成 pod 的调度。这是由
WaitForFirstConsumer
卷绑定模式指定的。延迟卷绑定使得调度器在为 PersistentVolumeClaim 选择一个合适的 PersistentVolume 时能考虑到所有 pod 的调度限制。
在web管理上看如下图所示:
2.4 建立 statefulset
#使用statefulset建立有状态mysql
cat>mysql8-statefulset.yaml<<EOF
apiVersion: v1
kind: Service
metadata:
name: mysql8
labels:
db: mysql8-ser-none
spec:
selector:
db: mysql8
ports:
- port: 3306
clusterIP: None
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql8
spec:
serviceName: "mysql8"
replicas: 2
selector:
matchLabels:
db: mysql8
template:
metadata:
labels:
db: mysql8
spec:
containers:
- name: mysql8
image: mysql:8.0.21
env:
- name: MYSQL_ROOT_PASSWORD
value: hua123
- name: MYSQL_DATABASE
value: /var/lib/mysql
args: ["--default-authentication-plugin=mysql_native_password","--character-set-server=utf8mb4","--collation-server=utf8mb4_unicode_ci"]
ports:
- containerPort: 3306
name: mysql8
volumeMounts:
- mountPath: /var/lib/mysql
name: data
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes:
- ReadWriteOnce
storageClassName: data-mysql8
resources:
requests:
storage: 30Gi
EOF
#cat mysql8-statefulset.yaml
kubectl apply -f mysql8-statefulset.yaml
2.5 查看运行情况
执行上面完成之后,通过web管理界面看一下
看一下StatefulSet情况,正常
看一下pvc情况:这些 PVC,都以“<PVC 名字 >-<StatefulSet 名字 >-< 编号 >”的方式命名,并且处于 Bound 状态。
再看一下pv情况,是不是绑定了
再看一下pod情况
都正常了,为了方便,我再安装一个nginx pod看一下情况怎样,查看一下它们的域名是什么,直接看hosts就行了
[root@vm82 myk8s]# kubectl exec mysql8-0 -c mysql8 -- grep mysql /etc/hosts
10.44.0.1 mysql8-0.mysql8.default.svc.cluster.local mysql8-0
[root@vm82 myk8s]#
[root@vm82 myk8s]# kubectl exec mysql8-1 -c mysql8 -- grep mysql /etc/hosts
10.44.0.2 mysql8-1.mysql8.default.svc.cluster.local mysql8-1
这样就得到每个pod的域名了,可以安装一个centos7 pod看一下能不能连接
# 使用 kubectl explain Pod.spec.containers 得倒docker exec -it命令在k8s pod的表达方式
cat>centos7.yaml<<EOF
apiVersion: v1
kind: Pod
metadata:
name: centos7
labels:
sys: centos7
spec:
containers:
- name: centos7
image: centos:7
stdin: true
tty: true
args: ["/bin/bash"]
EOF
kubectl apply -f centos7.yaml
kubectl get pod centos7
安装一个mysql客户端,测试一下是否能连接上
#登录进centos7容器中
kubectl exec -it centos7 -- bash
rpm -ih https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm
#只装一个客户端,测试一下mysql连接
yum install mysql-community-client -y
分别在mysql8-0和mysql8-1插入数据
[root@centos7 ~]# mysql -h mysql8-0.mysql8.default.svc.cluster.local -uroot -phua123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.0.21 MySQL Community Server - GPL
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CREATE DATABASE /*!32312 IF NOT EXISTS*/`hua` /*!40100 DEFAULT CHARACTER SET utf8 */;
Query OK, 1 row affected, 1 warning (0.16 sec)
mysql>
mysql> USE `hua`;
Database changed
mysql> DROP TABLE IF EXISTS `stu`;
Query OK, 0 rows affected, 1 warning (0.08 sec)
mysql>
mysql> CREATE TABLE `stu` (
-> `id` INT(6) UNSIGNED NOT NULL AUTO_INCREMENT,
-> `sid` VARCHAR(6) NOT NULL,
-> `name` VARCHAR(10) NOT NULL,
-> `yuWen` FLOAT DEFAULT '0',
-> `shuXue` FLOAT DEFAULT '0',
-> `sex` CHAR(2) NOT NULL DEFAULT '',
-> `zhiWei` CHAR(6) NOT NULL DEFAULT '',
-> PRIMARY KEY (`id`),
-> UNIQUE KEY `sid` (`sid`)
-> ) ENGINE=INNODB AUTO_INCREMENT=20 DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected, 1 warning (0.49 sec)
mysql> INSERT INTO `stu`(`id`,`sid`,`name`,`yuWen`,`shuXue`,`sex`,`zhiWei`) VALUES
-> (1,'1','',51,65,'',''),
-> (2,'2','',71,80,'','')
-> ;
Query OK, 2 rows affected (0.29 sec)
mysql> select * from stu;
+----+-----+------+-------+--------+-----+--------+
| id | sid | name | yuWen | shuXue | sex | zhiWei |
+----+-----+------+-------+--------+-----+--------+
| 1 | 1 | | 51 | 65 | | |
| 2 | 2 | | 71 | 80 | | |
+----+-----+------+-------+--------+-----+--------+
2 rows in set (0.00 sec)
mysql> exit
Bye
[root@centos7 ~]#
#现在向mysql8-1插入不同数据,设置数据库为hua1,表为stu1
mysql> CREATE DATABASE /*!32312 IF NOT EXISTS*/`hua1` /*!40100 DEFAULT CHARACTER SET utf8 */;
Query OK, 1 row affected, 1 warning (0.08 sec)
mysql> USE `hua1`;
Database changed
mysql> DROP TABLE IF EXISTS `stu1`;
Query OK, 0 rows affected, 1 warning (0.07 sec)
mysql>
mysql> CREATE TABLE `stu1` (
-> `id` INT(6) UNSIGNED NOT NULL AUTO_INCREMENT,
-> `sid` VARCHAR(6) NOT NULL,
-> `name` VARCHAR(10) NOT NULL,
-> `yuWen` FLOAT DEFAULT '0',
-> `shuXue` FLOAT DEFAULT '0',
-> `sex` CHAR(2) NOT NULL DEFAULT '',
-> `zhiWei` CHAR(6) NOT NULL DEFAULT '',
-> PRIMARY KEY (`id`),
-> UNIQUE KEY `sid` (`sid`)
-> ) ENGINE=INNODB AUTO_INCREMENT=20 DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected, 1 warning (0.52 sec)
mysql>
mysql> /*Data for the table `stu` */
mysql> INSERT INTO `stu1`(`id`,`sid`,`name`,`yuWen`,`shuXue`,`sex`,`zhiWei`) VALUES
-> (5,'5','',77,59,'',''),
-> (6,'6','',45,90,'','');
Query OK, 2 rows affected (0.10 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> select * from stu1;
+----+-----+------+-------+--------+-----+--------+
| id | sid | name | yuWen | shuXue | sex | zhiWei |
+----+-----+------+-------+--------+-----+--------+
| 5 | 5 | | 77 | 59 | | |
| 6 | 6 | | 45 | 90 | | |
+----+-----+------+-------+--------+-----+--------+
2 rows in set (0.00 sec)
mysql> \q;
Bye
[root@centos7 ~]#
#退出docker
[root@centos7 ~]# exit
exit
[root@vm82 myk8s]#
设置好了,我人为的把mysql-0删除看一下
#删除一个pod
[root@vm82 myk8s]# kubectl delete po mysql8-0
pod "mysql8-0" deleted
[root@vm82 myk8s]#
#发现k8s又自动重建一个mysql8-0
[root@vm82 myk8s]# kubectl get po
NAME READY STATUS RESTARTS AGE
centos7 1/1 Running 0 113m
mysql8-0 0/1 ContainerCreating 0 7s
mysql8-1 1/1 Running 0 144m
[root@vm82 myk8s]# kubectl get po
NAME READY STATUS RESTARTS AGE
centos7 1/1 Running 0 114m
mysql8-0 1/1 Running 0 74s
mysql8-1 1/1 Running 0 145m
现在我用登陆centos7容器,再用mysql连接一下,看一下里面的数据是执行sql查询,发现数据还在。
#登录进centos7容器中
kubectl exec -it centos7 -- bash
#登陆mysql
[root@centos7 /]# mysql -h mysql8-0.mysql8.default.svc.cluster.local -uroot -phua123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 10
Server version: 8.0.21 MySQL Community Server - GPL
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use hua;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from stu;
+----+-----+------+-------+--------+-----+--------+
| id | sid | name | yuWen | shuXue | sex | zhiWei |
+----+-----+------+-------+--------+-----+--------+
| 1 | 1 | | 51 | 65 | | |
| 2 | 2 | | 71 | 80 | | |
+----+-----+------+-------+--------+-----+--------+
2 rows in set (0.00 sec)
mysql> \q;
Bye
[root@centos7 /]# exit
exit
[root@vm82 myk8s]#