欢迎关注「WeiyiGeek」公众号
点击 👇 下方卡片 即可关注我哟!
设为「星标⭐」每天带你 基础入门 到 进阶实践 再到 放弃学习!
涉及 网络安全运维、应用开发、物联网IOT、学习路径 、个人感悟 等知识
专栏书写不易,如果您觉得这个专栏还不错的,请给这篇专栏 【点个赞、投个币、收个藏、关个注,转个发,留个言】(人间六大情),这将对我的肯定,谢谢!
本章目录:
-
前言简述
-
Docker 快速部署单实例 redis 数据库
-
Kubernetes 快速部署单实例 redis 数据库
-
Kubernetes 快速利用资源清单部署 redis 数据库集群
-
Kubernetes 中使用 Helm 快速部署 redis 多主多从集群
-
Kubernetes 快速部署 RedisInsight 工具连接Redis集群
原文地址: https://blog.weiyigeek.top/2022/4-24-653.html
前言简述
本章作者实践在 Docker 以及 kubernetes 环境中,快速部署生产环境中所使用的 Redis 内存数据库,帮助 devops工作者 以及 dev 开发者节省部署和开发时间。
如果你还不了解 Redis 数据库的朋友,可以参考我的【Redis学习之路
】系列笔记帮助你快速入门Redis数据库, 关注 WeiyiGeek 公众号回复【Redis学习之路汇总
】即可获得学习资料, 或访问如下链接https://www.weiyigeek.top/wechat.html?key=Redis学习之路汇总
Docker 快速部署单实例 redis 数据库
温馨提示:此处实践环境是使用Docker,若你没有安装Docker环境或者不了解的Docker容器的朋友,可以参考博主学习【Docker的系列笔记
】汇总, 关注 WeiyiGeek 公众号回复【Docker容器学习之路汇总
】即可获得学习资料:
https://www.weiyigeek.top/wechat.html?key=Docker容器学习之路汇总
Shell 命令示例:
mkdir -vp /app/redis/data
tee /app/redis/redis.conf <<'EOF'
# Author: WeiyiGeek
# blog: https://blog.weiyigeek.top
# 绑定任意接口、服务端口、后台运行。
bind 0.0.0.0
port 6379
# 容器里必须设置为no
daemonize no
supervised auto
# redis服务pid进程文件名
pidfile "/var/run/redis.pid"
# 关闭保护模式,并配置使用密码访问
protected-mode no
requirepass "123456"
# echo -e "weiyigeek"|sha256sum
# requirepass 097575a79efcd7ea7b1efa2bcda78a4fc7cbd0820736b2f2708e72c3d21f8b61
# 数据文件保存路径,rdb/AOF文件也保存在这里
dir "/data"
# 日志文件记录文件(notice / verbose)
# logfile "/logs/redis.log"
# loglevel verbose
# 最大客户端连接数
maxclients 10000
# 客户端连接空闲多久后断开连接,单位秒,0表示禁用
timeout 60
tcp-keepalive 60
# Redis 数据持久化(rdb/aof)配置
# RDB 文件名
dbfilename "dump.rdb"
# 数据自动保存脚本条件例如300s中有10key发生变化
save 300 10
save 60 10000
# 对RDB文件进行压缩,建议以(磁盘)空间换(CPU)时间。
rdbcompression yes
# 版本5的RDB有一个CRC64算法的校验和放在了文件的最后。这将使文件格式更加可靠。
rdbchecksum yes
# RDB自动触发策略是否启用,默认为yes
rdb-save-incremental-fsync yes
# AOF开启
appendonly yes
# AOF文件名
appendfilename "appendonly.aof"
# 可选值 always, everysec,no,建议设置为everysec
appendfsync everysec
# Redis风险命令重命名
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
rename-command FLUSHDB b840fc02d524045429941cc15f59e41cb7be6c53
rename-command FLUSHALL b840fc02d524045429941cc15f59e41cb7be6c54
rename-command EVAL b840fc02d524045429941cc15f59e41cb7be6c55
rename-command DEBUG b840fc02d524045429941cc15f59e41cb7be6c56
# rename-command SHUTDOWN SHUTDOWN
EOF
# docker 环境
$ docker run -d -p 6379:6379 \
-v /app/redis/redis.conf:/etc/redis/redis.conf \
-v /app/redis/data:/data \
--name redis-server redis:6.2.6-alpine3.15 redis-server /etc/redis/redis.conf
# nerdctl + containerd 环境
$ nerdctl run -d -p 6379:6379 \
-v /app/redis/redis.conf:/etc/redis/redis.conf \
-v /app/redis/data:/data \
--name redis-server redis:6.2.6-alpine3.15 redis-server /etc/redis/redis.conf
5e854a58087ae1bba5a661b2941474560cbecc37f54c7f4e7a28afbaed6aebf0
执行结果:
# 查看容器是否运行
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2919b8a9478a redis:6.2.6-alpine3.15 "docker-entrypoint.s…" 20 seconds ago Up 19 seconds 0.0.0.0:6379->6379/tcp redis-server
# 连接redis数据库是否正常
$ docker exec -it redis-server redis-cli -a 123456 ping
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
PONG
Kubernetes 快速部署单实例 redis 数据库
温馨提示:此处实践环境是使用Kubernetes集群,若你没有安装Kubernetes集群环境或者不了解的Kubernetes容器的朋友,可以参考博主学习【Kubernetes的系列笔记
】汇总, 关注 WeiyiGeek 公众号回复【Kubernetes学习之路汇总
】即可获得学习资料https://www.weiyigeek.top/wechat.html?key=Kubernetes学习之路汇总
温馨提示:集群中基于nfs的provisioner的动态持卷环境部署可参考此篇文章( https://blog.weiyigeek.top/2022/6-7-664.html )
步骤01.创建configMap 资源清单配置Redis.conf (此处还是采用上述 /app/redis/redis.conf 配置文件)
kubectl create configmap -n database redis-single-conf --from-file=/app/redis/redis.conf
kubectl get configmap -n database redis-single-conf
# NAME DATA AGE
# redis-single-conf 1 30s
步骤02.StatefulSet 与 SVC 资源清单:
tee redis-single-server.yaml <<'EOF'
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: __APPNAME__
namespace: __APPNAMESPACE__
annotations:
description: redis-single
spec:
replicas: 1
serviceName: __APPNAME__
selector:
matchLabels:
app: __APPNAME__
template:
metadata:
labels:
app: __APPNAME__
spec:
# 容器优化
initContainers:
- name: sysctl
image: alpine:3.15.4
imagePullPolicy: IfNotPresent
command:
- sh
- -c
- |
mount -o remount rw /proc/sys
sysctl -w net.core.somaxconn=10000
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.ipv4.ip_local_port_range="1024 65535"
sysctl -w fs.file-max=1048576
sysctl -w fs.inotify.max_user_instances=16384
sysctl -w fs.inotify.max_user_watches=524288
sysctl -w fs.inotify.max_queued_events=16384
echo never > /sys/kernel/mm/transparent_hugepage/enabled
securityContext:
privileged: true
containers:
- name: redis
image: redis:6.2.6-alpine3.15
imagePullPolicy: IfNotPresent
ports:
- containerPort: 6379
name: server
command: [ "redis-server", "/conf/redis.conf"]
volumeMounts:
# 从configmap获取的配置文件,挂载到指定文件中【https://blog.weiyigeek.top】.
- name: conf
mountPath: /conf/redis.conf
subPath: redis.conf
- name: data
mountPath: /data
# - name: logs
# mountPath: /logs
# 时区设置
- name: timezone
mountPath: /etc/localtime
volumes:
- name: conf
# 配置文件采用configMap
configMap:
name: redis-single-conf
defaultMode: 0755
# 日志采用hostPath卷
- name: logs
hostPath:
type: DirectoryOrCreate
path: /app/redis/logs
# 时区定义
- name: timezone
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "__STORAGECLASSNAME__"
resources:
requests:
storage: 2Gi
---
apiVersion: v1
kind: Service
metadata:
name: __APPNAME__
namespace: __APPNAMESPACE__
spec:
type: ClusterIP
ports:
- port: 6379
targetPort: 6379
name: tcp
selector:
app: __APPNAME__
EOF
# 注意 替换 storageClassName
sed -i -e 's/__APPNAME__/redis-single/g' -e 's/__APPNAMESPACE__/database/g' -e 's/__STORAGECLASSNAME__/managed-nfs-storage/' redis-single-server.yaml
步骤 03.部署资源清单到Kubernetes集群
kubectl apply -f redis-single-server.yaml
# statefulset.apps/redis-single created
# service/redis-single created
步骤 04.验证部署应用资源
kubectl get pod -n database -l app=redis-single
# NAME READY STATUS RESTARTS AGE
# redis-single-0 1/1 Running 0 16m
kubectl get svc -n database redis-single
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# redis-single ClusterIP 11.19.196.169 <none> 6379/TCP 15m
# 连接到redis数据库
$ telnet 11.19.196.169 6379
# Trying 11.19.196.169...
# Connected to 11.19.196.169.
# Escape character is '^]'.
# auth 123456
# +OK
# ping
# +PONG
# info
# $4241
# # Server
# redis_version:6.2.6
# redis_git_sha1:00000000
# redis_git_dirty:0
# redis_build_id:63421500bb103677
# redis_mode:standalone
# os:Linux 4.20.13-1.el7.elrepo.x86_64 x86_64
# arch_bits:64
# multiplexing_api:epoll
# atomicvar_api:atomic-builtin
# gcc_version:10.3.1
# process_id:1
# process_supervised:no
# run_id:677b6d0188b564670f26ce47154b70c8aa6c3f04
# tcp_port:6379
# server_time_usec:1650962303340701
温馨提示: 集群中的其它应用,我们可以通过svc的地址或者集群dns域名(redis-single.database
-推荐)来访问该pod。
Kubernetes 快速利用资源清单部署 redis 数据库集群
步骤 01.创建 redis 配置文件使用 configmap 方式进行挂载。
tee redis-cluster-configmap.yaml << 'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-cluster
namespace: database
data:
update-node.sh: |
#!/bin/sh
CLUSTER_CONFIG="/data/nodes.conf"
if [ -f ${CLUSTER_CONFIG} ]; then
if [ -z "${POD_IP}" ]; then
echo "Unable to determine Pod IP address!"
exit 1
fi
echo "Updating my IP to ${POD_IP} in ${CLUSTER_CONFIG}"
sed -i.bak -e '/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/'${POD_IP}'/' ${CLUSTER_CONFIG}
fi
exec "$@"
redis.conf: |+
port 6379
protected-mode no
maxclients 32768
masterauth weiyigeek.top
requirepass weiyigeek.top
# 数据目录
dir /data
# 开启 RDB 持久化&触发策略
dbfilename dump.rdb
rdb-save-incremental-fsync yes
# 开启AOF持久化以及每秒钟同步一次折中的方案
appendonly yes
appendfilename appendonly.aof
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 128mb
aof-load-truncated yes
aof-use-rdb-preamble yes
aof-rewrite-incremental-fsync yes
# 集群模式打开
cluster-enabled yes
cluster-config-file /data/nodes.conf
cluster-node-timeout 5000
# 当负责一个插槽的主库下线且没有相应的从库进行故障恢复时集群仍然可用
cluster-require-full-coverage no
# 只有当一个主节点至少拥有其他给定数量个处于正常工作中的从节点的时候,才会分配从节点给集群中孤立的主节点
cluster-migration-barrier 1
# 副本服务陈旧数据
replica-serve-stale-data yes
replica-read-only no
replica-priority 100
# 复制同步策略
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
# 慢查询日志
slowlog-log-slower-than 10000
slowlog-max-len 128
# 内存策略
maxmemory-policy allkeys-lru
EOF
温馨提示:masterauth 与 requirepass 认证密码需要根据实际情况进行更改,此外我设置为了我的主页地址。
步骤 02.Redis 集群 SVC 以及 STS 资源清单。
tee redis-cluster-deploy.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
namespace: database
name: redis-cluster
labels:
app: redis-cluster
spec:
clusterIP: None
ports:
- port: 6379
targetPort: 6379
name: client
- port: 16379
targetPort: 16379
name: gossip
selector:
app: redis-cluster
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
namespace: database
name: redis-cluster
labels:
app: redis-cluster
spec:
serviceName: redis-cluster
replicas: 6
selector:
matchLabels:
app: redis-cluster
template:
metadata:
labels:
app: redis-cluster
spec:
affinity:
nodeAffinity: # node亲和性
requiredDuringSchedulingIgnoredDuringExecution: # 硬策略,调度在指定标签的节点中
nodeSelectorTerms:
- matchExpressions:
- key: node
operator: In
values:
- app
podAntiAffinity: # Pod反亲和性
preferredDuringSchedulingIgnoredDuringExecution: # 软策略,使Pod分布在不同的节点上
- weight: 1 # 权重,有多个策略通过权重控制调度
podAffinityTerm:
topologyKey: app.kubernetes.io/name # 通过app.kubernetes.io/name作为域调度
labelSelector:
matchExpressions:
- key: node
operator: In
values:
- app
containers:
- name: redis
image: redis:6.2.7-alpine
ports:
- containerPort: 6379
name: client
- containerPort: 16379
name: gossip
command: ["/etc/redis/update-node.sh", "redis-server", "/etc/redis/redis.conf"]
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: conf
mountPath: /etc/redis/
readOnly: false
- name: data
mountPath: /data
readOnly: false
- name: timezone
mountPath: /etc/localtime # 在Pod中时区设置(挂载主机的时区)
volumes:
- name: conf
configMap:
name: redis-cluster
defaultMode: 0755
- name: timezone
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: data
spec:
storageClassName: nfs-dev
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
EOF
步骤 03.使用 kubectl apply 部署资源清单,并查看部署结果以及Pod IP,为集群初始化做准备。
# (1) 名称空间创建
$ kubectl create namespace database
# (2) 按照资源清单StatefulSet控制器创建Pod
$ ls
redis-cluster-configmap.yaml redis-cluster-deploy.yaml
$ kubectl create -f redis-cluster-configmap.yaml
$ kubectl create -f redis-cluster-deploy.yaml
# (3) 查看应用的 statefulsets,pod,svc 等信息
$ kubectl get statefulsets,pod,svc -n database
NAME READY AGE
statefulset.apps/redis-cluster 6/6 13m
NAME READY STATUS RESTARTS AGE
pod/redis-cluster-0 1/1 Running 0 13m
pod/redis-cluster-1 1/1 Running 0 12m
pod/redis-cluster-2 1/1 Running 0 12m
pod/redis-cluster-3 1/1 Running 0 10m
pod/redis-cluster-4 1/1 Running 0 9m19s
pod/redis-cluster-5 1/1 Running 0 9m16s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-cluster ClusterIP None <none> 6379/TCP,16379/TCP 13m
# (4) 获取 Redis 集群 6 个节点 Pod 的 ip 地址
kubectl get pod -n database -l app=redis-cluster -o jsonpath='{ range.items [*]}{.status.podIP}:6379 '| sed "s# :6379 ##g"
# 10.66.35.66:6379 10.66.237.142:6379 10.66.55.76:6379 10.66.53.96:6379 10.66.35.92:6379 10.66.55.77:6379
步骤 04.进入到第一个Pod 终端中进行初始化 Redis 集群操作,验证Redis Cluster集群
# (1) 进入redis-cluster-0 pod中的shell
kubectl exec -n database -it redis-cluster-0 sh
# (2) 进行集群初始化,运行以下命令并输入'yes'接受配置。我们将看到前三个节点将被选择为主节点,后三个节点将被选择为从节点。
/data $ redis-cli -a weiyigeek.top --cluster create --cluster-replicas 1 10.66.35.66:6379 10.66.237.142:6379 10.66.55.76:6379 10.66.53.96:6379 10.66.35.92:6379 10.66.55.77:6379
# >>> Performing hash slots allocation on 6 nodes...
# Master[0] -> Slots 0 - 5460
# Master[1] -> Slots 5461 - 10922
# Master[2] -> Slots 10923 - 16383
# Adding replica 10.66.35.92:6379 to 10.66.35.66:6379
# Adding replica 10.66.55.77:6379 to 10.66.237.142:6379
# Adding replica 10.66.53.96:6379 to 10.66.55.76:6379
# M: b13a58f799e058dd9afba221f25a8bd53ae05969 10.66.35.66:6379
# slots:[0-5460] (5461 slots) master
# M: 729af3b961a978ad15263ef96439177207be7821 10.66.237.142:6379
# slots:[5461-10922] (5462 slots) master
# M: 50c3d31e7277ce339752a980f14c03d180e2f98a 10.66.55.76:6379
# slots:[10923-16383] (5461 slots) master
# S: ce02a1eb7efbd4de37383f46c300bb883dcd16b4 10.66.53.96:6379
# replicates 50c3d31e7277ce339752a980f14c03d180e2f98a
# S: 298d5e8528c38f687c9894b9974eb5368555c652 10.66.35.92:6379
# replicates b13a58f799e058dd9afba221f25a8bd53ae05969
# S: f7ecf69d99129fba7737f8ae9a8f172af19a7b72 10.66.55.77:6379
# replicates 729af3b961a978ad15263ef96439177207be7821
# Can I set the above configuration? (type 'yes' to accept): yes # 此处输入 yes 进行上述卡槽设置。
# >>> Nodes configuration updated
# >>> Assign a different config epoch to each node
# >>> Sending CLUSTER MEET messages to join the cluster
# Waiting for the cluster to join
# .
# >>> Performing Cluster Check (using node 10.66.35.66:6379)
# M: b13a58f799e058dd9afba221f25a8bd53ae05969 10.66.35.66:6379
# slots:[0-5460] (5461 slots) master
# 1 additional replica(s)
# S: f7ecf69d99129fba7737f8ae9a8f172af19a7b72 10.66.55.77:6379
# slots: (0 slots) slave
# replicates 729af3b961a978ad15263ef96439177207be7821
# M: 50c3d31e7277ce339752a980f14c03d180e2f98a 10.66.55.76:6379
# slots:[10923-16383] (5461 slots) master
# 1 additional replica(s)
# S: ce02a1eb7efbd4de37383f46c300bb883dcd16b4 10.66.53.96:6379
# slots: (0 slots) slave
# replicates 50c3d31e7277ce339752a980f14c03d180e2f98a
# M: 729af3b961a978ad15263ef96439177207be7821 10.66.237.142:6379
# slots:[5461-10922] (5462 slots) master
# 1 additional replica(s)
# S: 298d5e8528c38f687c9894b9974eb5368555c652 10.66.35.92:6379
# slots: (0 slots) slave
# replicates b13a58f799e058dd9afba221f25a8bd53ae05969
# [OK] All nodes agree about slots configuration.
# >>> Check for open slots...
# >>> Check slots coverage...
# [OK] All 16384 slots covered.
# (3) 集群验证
# 此处我使用集群svc服务名称进行访问 redis-cluster.database.svc.cluster.test。
/data $ redis-cli -c -h redis-cluster.database.svc.cluster.test -a weiyigeek.top
# 查看集群信息
redis-cluster.database.svc.cluster.test:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
....
# 查看集群节点
redis-cluster.database.svc.cluster.test:6379> CLUSTER NODES
f7ecf69d99129fba7737f8ae9a8f172af19a7b72 10.66.55.77:6379@16379 slave 729af3b961a978ad15263ef96439177207be7821 0 1665234666115 2 connected
.......
ce02a1eb7efbd4de37383f46c300bb883dcd16b4 10.66.53.96:6379@16379 slave 50c3d31e7277ce339752a980f14c03d180e2f98a 0 1665234665110 3 connected
温馨提示:在初始化 redis 集群时必须使用 ip 地址,如果使用域名会报如下错误。温馨提示:当应用连接 redis 集群时使用 pod 的域名, 格式为 svc名字.ns名字.svc.cluster.local
$ nslookup redis-cluster.database.svc.cluster.test
Server: 10.96.0.10
Address: 10.96.0.10:53
Name: redis-cluster.database.svc.cluster.test
Address: 10.66.35.66
Name: redis-cluster.database.svc.cluster.test
Address: 10.66.35.92
Name: redis-cluster.database.svc.cluster.test
Address: 10.66.55.76
Name: redis-cluster.database.svc.cluster.test
Address: 10.66.53.96
Name: redis-cluster.database.svc.cluster.test
Address: 10.66.237.142
Name: redis-cluster.database.svc.cluster.test
Address: 10.66.55.77
步骤 05.使用redis cluster进行KV数据插入并查看数据。
redis-cluster.database.svc.cluster.test:6379> set domain weiyigeek.top EX 10
-> Redirected to slot [5449] located at 10.66.35.66:6379
OK
10.66.35.66:6379> get domain
"weiyigeek.top"
10.66.35.66:6379> get domain
"weiyigeek.top"
10.66.35.66:6379> get domain # 10秒后到期,则key被销毁
(nil
温馨提示:如果整个 redis 集群的 pod 全部都挂掉了,pod自动拉起后集群不可用,需要重建集群,此时有两种重建集群方法。
-
方式1.重新开始(不到万不得已不建议这样做)
1.删除 redis 集群所有的资源,然后重新创建 redis 集群2.删除 redis 集群中所有的 pvc(pv)3.删除 redis 集群中 pod 对应的 nfs 持久化存储目录4.重新创建 redis 集群资源并进行初始化
-
方式2.在原有 redis 集群的基础上进行修复。
1.将 redis 集群中资源控制器的副本数设置为 02.找到 redis 集群中 pod 对应的 nfs 持久化存储目录后删除 nodes.conf
2.找到 redis 集群中 pod 对应的 nfs 持久化存储目录后删除 nodes.conf
3.重新创建 redis 集群资源
kubectl apply -f redis-cluster-deploy.yaml
导出数据后,然后重新初始化集群
/storage/dev/pvc/local# find . -name nodes.conf
./database-data-redis-cluster-1-pvc-febdd490-6dbf-4084-860f-1d81602f4ca0/nodes.conf
./database-data-redis-cluster-3-pvc-08d6d4b4-5290-4710-b31e-efb5e8d8481e/nodes.conf
./database-data-redis-cluster-5-pvc-96db8931-2106-4ac1-9c41-ec8888e99024/nodes.conf
./database-data-redis-cluster-4-pvc-c67c4649-ff0f-4767-ae75-3875653886c5/nodes.conf
./database-data-redis-cluster-0-pvc-e6b1f1bd-75b0-4777-ab93-993a2e6e5927/nodes.conf
./database-data-redis-cluster-2-pvc-d078a7d3-cf0c-4b1e-acbe-59086089fb0d/nodes.conf
Kubernetes 中使用 Helm 快速部署 redis 多主多从集群
描述:我们可以使用 Bitnami helm chart 在 K8S 中一键部署 Redis cluster 多主多从。项目地址:https://github.com/bitnami/charts/tree/master/bitnami/redis-cluster
环境依赖:
-
Kubernetes 1.19+
-
Helm 3.2.0+
-
PV provisioner support in the underlying infrastructure
温馨提示:1.集群中基于nfs的provisioner的动态持卷环境部署可参考此篇文章( https://blog.weiyigeek.top/2022/6-7-664.html )
步骤 01.添加 Bitnami chart 仓库,查看可用 redis-cluster 的 chart 及其对应版本,拉取部署所需的 yaml 文件到本地。
$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm3 search repo bitnami/redis-cluster -l
$ helm3 pull bitnami/redis-cluster --version 8.2.4 --untar
步骤 02.修改 redis-cluster 图表的 values.yaml 文件
vim redis-cluster/values.yaml
# 1.修改 clusterDomain Kubernetes Cluster Domain 此处由于我们的集群名称非 cluster.local 所以修改如下
clusterDomain: cluster.test
.....
# 2.集群所需redis镜像,此处我已经上传内部harbor中,则需要根据实际请个改变
image:
registry: harbor.weiyigeek.top
repository: library/redis-cluster
tag: 7.0.5-debian-11-r0
.....
# 3.设置 Pod 安全上下文
podSecurityContext:
enabled: true
fsGroup: 1001
runAsUser: 1001
sysctls: []
.....
# 4.此处svc采用ClusterIP如果采用NodePort需要更改暴露端口 30000 - 32000
service:
ports:
redis: 6379
nodePorts:
redis: 31379
type: ClusterIP
.....
# 5.数据 persistence 持久化配置,此处我采用了NFS动态逻辑卷。
persistence:
path: /bitnami/redis/data
storageClass: "nfs-local"
accessModes:
- ReadWriteOnce
size: 8Gi
.....
# 6.设置 Redis statefulset parameters 资源限额
redis:
resources:
limits:
cpu: 1000m
memory: 4Gi
requests: {}
.....
# 7.Redis Cluster 设置,此处我设置3主3从,一共6个节点
# 如果 nodes 为 6 replicas 为 1 表示每个master节点有一个副本
# nodes = numberOfMasterNodes + numberOfMasterNodes * replicas
cluster:
init: true
nodes: 6
replicas: 1
.....
# 8.启用 Redis Prometheus Exporter Metrics
metrics:
enabled: true
image:
registry: harbor.weiyigeek.top
repository: library/redis-exporter
tag: 1.44.0-debian-11-r8
.....
# 9.启用并使用sysctlImage配置值设置特权initContainer
sysctlImage:
enabled: true
registry: harbor.weiyigeek.top
repository: library/bitnami-shell
tag: 11-debian-11-r37
command:
- /bin/sh
- -c
- |-
sysctl -w net.core.somaxconn=10000
echo never > /host-sys/kernel/mm/transparent_hugepage/enabled
mountHostSys: true
步骤 03.为了加快拉取速度我将docker hub上的指定所需镜像拉取到本地Harbor仓库中。
# redis-cluster:7.0.5-debian-11-r0
docker pull bitnami/redis-cluster:7.0.5-debian-11-r0
docker tag bitnami/redis-cluster:7.0.5-debian-11-r0 harbor.weiyigeek.top/library/redis-cluster:7.0.5-debian-11-r0
docker push harbor.weiyigeek.top/library/redis-cluster:7.0.5-debian-11-r0
# redis-exporter & bitnami-shell
docker pull bitnami/redis-exporter:1.44.0-debian-11-r8; docker pull bitnami/bitnami-shell:11-debian-11-r37
docker tag bitnami/redis-exporter:1.44.0-debian-11-r8 harbor.weiyigeek.top/library/redis-exporter:1.44.0-debian-11-r8
docker tag bitnami/bitnami-shell:11-debian-11-r37 harbor.weiyigeek.top/library/bitnami-shell:11-debian-11-r37
docker push harbor.weiyigeek.top/library/redis-exporter:1.44.0-debian-11-r8
docker push harbor.weiyigeek.top/library/bitnami-shell:11-debian-11-r37
步骤 04.开始安装修改后的 redis-cluster 图表到 database 名称空间中,并查看资源部署情况。
helm3 install redis ./redis-cluster --set password=weiyigeek.top --namespace database --create-namespace --debug
helm3 list -n database
# NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
# redis database 1 2022-10-09 17:35:18.530202622 +0800 CST deployed redis-cluster-8.2.4 7.0.5
kubectl get sts,pod,svc -n database -l app.kubernetes.io/name=redis-cluster
# NAME READY AGE
# statefulset.apps/redis-redis-cluster 6/6 4m4s
# NAME READY STATUS RESTARTS AGE
# pod/redis-redis-cluster-0 2/2 Running 1 (3m3s ago) 4m4s
# pod/redis-redis-cluster-1 2/2 Running 1 (3m3s ago) 4m4s
# pod/redis-redis-cluster-2 2/2 Running 2 (3m45s ago) 4m4s
# pod/redis-redis-cluster-3 2/2 Running 1 (3m ago) 4m4s
# pod/redis-redis-cluster-4 2/2 Running 1 (3m3s ago) 4m4s
# pod/redis-redis-cluster-5 2/2 Running 1 (3m ago) 4m4s
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# service/redis-redis-cluster ClusterIP 10.105.252.15 <none> 6379/TCP 4m4s
# service/redis-redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 4m5s
# service/redis-redis-cluster-metrics ClusterIP 10.96.154.84 <none> 9121/TCP 4m4s
步骤 05.验证使用helm部署的Redis集群
# 1.查询设置或者机生成的 Redis 密码
export REDIS_PASSWORD=$(kubectl get secret --namespace "database" redis-redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
# 2.创建运行Redis客户端的pod:
kubectl run --namespace database redis-redis-cluster-client --rm --tty -i --restart='Never' \
--env REDIS_PASSWORD=$REDIS_PASSWORD \
--image bitnami/redis-cluster:7.0.5-debian-11-r0 -- bash
# 3.在Pod中使用shell连接到集群
redis-cli -c -h redis-redis-cluster -a $REDIS_PASSWORD
# 集群信息
redis-redis-cluster:6379> CLUSTER INFO
# cluster_state:ok
# 节点信息
redis-redis-cluster:6379> CLUSTER NODES
# 数据插入测试
redis-redis-cluster:6379> set name weiyigeek
OK
10.66.237.149:6379> set domain weiyigeek.top
OK
10.66.35.100:6379> set blog blog.weiyigeek.top
OK
10.66.237.149:6379> keys *
1) "name"
2) "blog"
10.66.237.149:6379> get name
"weiyigeek"
至此,快速使用helm安装多主多从redis集群完毕!
Kubernetes 快速部署 RedisInsight 工具连接Redis集群
描述:RedisInsight 是 Redis 官方推荐的可视化工具,其功能强大,支持Redis集群连接。
参考地址:https://docs.redis.com/latest/ri/installing/install-k8s/
参考文章: 【使用RedisInsight工具对Redis集群CURD操作及数据可视化和性能监控】https://blog.weiyigeek.top/2022/9-20-686.html
步骤 01.集群中快速 redisinsight 部署资源清单。
tee redisinsight-1.13.0.yaml <<'EOF'
# RedisInsight service with name 'redisinsight-service'
apiVersion: v1
kind: Service
metadata:
name: redisinsight-service
namespace: dev
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 8001
selector:
app: redisinsight
---
# RedisInsight deployment with name 'redisinsight'
apiVersion: apps/v1
kind: Deployment
metadata:
name: redisinsight # deployment name
namespace: dev
labels:
app: redisinsight # deployment label
spec:
replicas: 1 # a single replica pod
strategy:
type: Recreate
selector:
matchLabels:
app: redisinsight # which pods is the deployment managing, as defined by the pod template
template: # pod template
metadata:
labels:
app: redisinsight # label for pod/s
spec:
volumes:
- name: db
emptyDir: {} # node-ephemeral volume https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
containers:
- name: redisinsight # Container name (DNS_LABEL, unique)
image: redislabs/redisinsight:1.13.0 # Hub Image
imagePullPolicy: IfNotPresent # Pull Policy
env:
- name: RIHOST
value: "0.0.0.0"
- name: RIPORT
value: "8001"
volumeMounts:
- name: db
mountPath: /db
ports:
- containerPort: 8001 # exposed container port and protocol
protocol: TCP
livenessProbe:
httpGet:
path : /healthcheck/ # exposed RI endpoint for healthcheck
port: 8001 # exposed container port
initialDelaySeconds: 5 # number of seconds to wait after the container starts to perform liveness probe
periodSeconds: 5 # period in seconds after which liveness probe is performed
failureThreshold: 1 # number of liveness probe failures after which container restarts
EOF
步骤 02.部署命令
kubectl create ns dev
kubectl apply -f redisinsight-1.13.0.yaml
步骤 03.验证服务(你可按照需求使用nodeport或者ingress进行暴露端口)
$ kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
redisinsight-5c4f954694-jc5cd 1/1 Running 0 95s
$ kubectl -n dev port-forward deployment/redisinsight --address 0.0.0.0 30081:8001
Forwarding from 0.0.0.0:30081 -> 8001
Handling connection for 30081
Handling connection for 30081
Handling connection for 30081
步骤 04.浏览器访问 节点IP:30081 即可访问 redisinsight UI 界面。
至此完毕。
本文至此完毕,更多技术文章,尽情期待下一章节!
原文地址: https://blog.weiyigeek.top/2022/4-24-653.html
亲,文章都看完了,不关注一下吗?点击 👆 卡片即可关注我哟!
往期发布文章
大神之路-起始篇 | 第1章.计算机科学导论之【基础绪论】学习笔记
大神之路-起始篇 | 第2章.计算机科学导论之【数字系统】学习笔记
大神之路-起始篇 | 第3章.计算机科学导论之【数据存储】学习笔记
大神之路-起始篇 | 第4章.计算机科学导论之【数据运算】学习笔记
大神之路-起始篇 | 第5章.计算机科学导论之【计算机组成】学习笔记
大神之路-起始篇 | 第6章.计算机科学导论之【计算机网络】学习笔记
企业运维 | MySQL关系型数据库在Docker与Kubernetes容器环境中快速搭建部署主从实践
欢迎各位志同道合的朋友一起学习交流,如文章有误请在下方留下您宝贵的经验知识,个人邮箱地址 【master#weiyigeek.top 】或者个人公众号【 WeiyiGeek 】联系我。
WeiyiGeek Blog 个人博客 - 为了能到远方,脚下的每一步都不能少!
个人主页: 【 https://weiyigeek.top】
博客地址: 【 https://blog.weiyigeek.top 】
学习交流群:【 https://weiyigeek.top/visit.html 】
更多网络安全、系统运维、应用开发、物联网实战、全栈文章,尽在【个人博客 - https://blog.weiyigeek.top 】站点,谢谢支持!
帅哥、美女、大佬们点个【赞+在看】吧! 👇
👇👇👇 点击下方【"阅读原文"】,即可获取更多有趣的知识!