Redis单例部署

1. 概述

之前一直使用Redis,最近遇到一些问题,整理Redis的单例安装部署过程,记录如下。

2. 参考

  1. 链接: Install Redis
  2. 链接: github-Redis
  3. 链接: gitcode-Redis

3. 环境

  • 虚机Virtual-Machine-203
  • Ubuntu 22.04(Centos7.9也可以)
  • Redis 7.2.5

4. 部署

4.1 操作系统

4.1.1 修改系统参数

  • 添加以下系统参数(必须)
cat >> /etc/sysctl.conf << EOF
# Redis 必须使用参数
#定义了系统中每一个端口最大的监听队列的长度,默认4096 可调整到8192/16384/32768
net.core.somaxconn = 32768

# Redis 必须使用参数
# 1表示内核允许分配所有的物理内存,而不管当前的内存状态如何,默认0。
vm.overcommit_memory = 1

# Redis 建议使用参数
# 物理内存使用90%,才开始使用swap,默认60,也可以设置为0,优先使用100%物理内存
vm.swappiness = 10

# 大于100将更积极的回收cache,默认100,
vm.vfs_cache_pressure = 100

EOF
  • 添加以下系统参数(可选)
cat >> /etc/sysctl.conf << EOF
# 系统调优参数
# 允许存在time_wait状态的最大数值,超过则立刻被清除并且警告 
# 默认是262144, ubuntu默认8192,防止过多的time_wait导致端口资源被耗尽
net.ipv4.tcp_max_tw_buckets = 30000

# 开启SYN Cookies,当出现SYN 等待队列溢出时,启用cookies 来处理。
net.ipv4.tcp_syncookies = 1

# 该参数决定了,每个网络接口接收数据包的速率比内核处理这些包的速率快时,允许送到队列的数据包的最大数目,默认1000
net.core.netdev_max_backlog = 32768

# 表示SYN队列的长度,默认2048
net.ipv4.tcp_max_syn_backlog = 32768

# 建立连接syn+ack 与syn 包重试次数
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2

EOF
  • 参数生效
sysctl -p

4.1.2 关闭透明大页内存

  • 关闭透明大页内存动态分配,需要关闭让 redis 或 mongo 负责内存管理
cat >> /etc/rc.local << EOF
# Redis Mongo建议参数
# Disable Transparent Huge Pages, redis and mongo configure
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
EOF
  • 加载生效
source /etc/rc.local

4.1.3 修改系统限制

cp /etc/security/limits.conf /etc/security/limits.conf.bak
cat>>/etc/security/limits.conf <<EOF
* soft nproc 655350
* hard nproc 655350
* soft nofile 655350
* hard nofile 655350

# ubuntu 需要针对账号设置
root soft nproc 655350
root hard nproc 655350
root soft nofile 655350
root hard nofile 655350
EOF
  • 重启或重新登录生效
  • 修改当前设置生效
ulimit -n 655350 
ulimit -u 655350 

4.2 安装Redis

4.2.1 下载Redis

  • 从官网或 github下载,需要看一下版本,选择没有重大安全问题的版本
    • https://github.com/redis/redis/releases
    • http://download.redis.io/releases/redis-7.2.5.tar.gz
wget http://download.redis.io/releases/redis-7.2.5.tar.gz

4.2.2 创建redis账号

useradd -s /sbin/nologin redis

4.2.3 添加Redis环境变量

  • 确定Redis安装路径,例如/usr/local/redis7.2.5
  • 增加环境变量 REDIS_HOME ,不要与已有路径冲突
cat >>/etc/profile<<EOF

export REDIS_HOME=/usr/local/redis7.2.5
PATH=\$REDIS_HOME/bin:\$PATH
export PATH
EOF
  • 加载生效,查看路径
source /etc/profile && echo $PATH

4.2.4 创建Redis使用目录

  • 创建Redis使用目录
    • /usr/local/redis7.2.5(REDIS_HOME):程序存放目录
    • /var/log/redis:日志存放目录
    • /var/run/redis:服务PID存放目录
    • /etc/redis:配置文件存放目录(可选),当前默认存放在“程序存放目录”下
mkdir -p ${REDIS_HOME} && chown -R redis:redis ${REDIS_HOME}
mkdir -p /var/log/redis && chown redis:redis /var/log/redis
mkdir -p /var/run/redis && chown -R redis:redis /var/run/redis
# mkdir -p /etc/redis && chown -R redis:redis /etc/redis

4.2.5 安装Redis

安装可以根据Redis服务启动管理,选择安装方式。可提供以下几种方式,供选择:

  1. 使用systemctl命令管理(Centos7、Ubuntu),编译安装时指定 USE_SYSTEMD=yes 选项,配置systemd服务管理文件时Type使用notify
  2. 使用systemctl命令管理(Centos7、Ubuntu),编译安装,配置systemd服务管理文件时Type使用simple
  3. 使用service命令管理(Centos6),编译安装,配置/etc/init.d下启动文件
  • Ubuntu支持systemd方式编译安装,能够支持Type=notify
    • PREFIX 指定安装路径
    • USE_SYSTEMD 支持systemd
# Ubuntu 安装
tar -zxvf redis-7.2.5.tar.gz && cd redis-7.2.5
# 安装依赖包
apt install -y libsystemd-dev pkg-config
# 可以使用支持systemd方式编译安装,能够支持Type=notify
make PREFIX=${REDIS_HOME} USE_SYSTEMD=yes install
  • Centos支持systemd方式编译安装,能够支持Type=notify
    • PREFIX 指定安装路径
    • USE_SYSTEMD 支持systemd
# Centos 安装
tar -zxvf redis-7.2.5.tar.gz && cd redis-7.2.5
# 安装依赖包
yum install -y systemd-notify-devel systemd-devel
# 可以使用支持systemd方式编译安装,能够支持Type=notify
make PREFIX=${REDIS_HOME} USE_SYSTEMD=yes install
  • 其他编译安装,可用于service命令(如Centos6)或systemd的simple
    • PREFIX 指定安装路径
tar -zxvf redis-6.2.9.tar.gz && cd redis-6.2.9
make PREFIX=${REDIS_HOME} install
  • 验证安装版本
${REDIS_HOME}/bin/redis-server --version

4.2.6 手动修改配置文件(可跳过,直接使用4.2.7命令修改

  • 拷贝配置文件到安装路径下并修改。
cp ./redis.conf ${REDIS_HOME}/ && cd ${REDIS_HOME}
  • 修改监听地址,IPv4和IPv6均监听
#bind 127.0.0.1 -::1
bind * -::*
  • 根据需要修改监听端口
# Accept connections on the specified port, default is 6379 (IANA #815344).
# If port 0 is specified Redis will not listen on a TCP socket.
port 6379
  • 修改监听队列长度,小于sysctl.conf文件中的somaxconn 设置
# In high requests-per-second environments you need a high backlog in order
# to avoid slow clients connection issues. Note that the Linux kernel
# will silently truncate it to the value of /proc/sys/net/core/somaxconn so
# make sure to raise both the value of somaxconn and tcp_max_syn_backlog
# in order to get the desired effect.
#tcp-backlog 511
tcp-backlog 4096
  • timeout保持不变,配合应用程序保持长连接。
    注意它不是超时连接,它是空闲后等待多久后关闭。
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 0
  • tcp-keepalive保持不变
    服务端于探测客户端的检测时间周期。可根据需要调整
tcp-keepalive 300
  • 启动后台运行
#daemonize no
daemonize yes
  • 修改pid文件路径
#pidfile /var/run/redis_6379.pid
pidfile /var/run/redis/redis_6379.pid
  • 修改日志文件路径
#logfile ""
logfile "/var/log/redis/redis_6379.log"
  • 修改snapshotting配置
    需要综合考虑数据是否允许丢失和写入rdb的频率。
    这里是允许数据丢失的,而且变更的频次不高,所以只修改了900秒内有500个变更的值
################################ SNAPSHOTTING  ################################

# Save the DB to disk.
#
# You can set these explicitly by uncommenting the following line.
#
# save 3600 1 300 100 60 10000
save 3600 1 900 500 60 10000
  • 修改了dump.rdb的文件名称
# The filename where to dump the DB
 dbfilename dump_6379.rdb
  • 修改了dump.rdb的存储路径
    注意路径是否有足够的磁盘空间。最合理方式应该存储在单独的数据分区,创建目录并给redis账号赋权
# dir ./
dir /usr/local/redis/
  • 修改连接密码
# requirepass foobared
 requirepass yourpassword
  • 重命名危险命令
    • config命令使用别名
    • 其它命令被禁用
# rename-command CONFIG ""
 rename-command CONFIG "kkconfig"
 rename-command FLUSHALL ""
 rename-command FLUSHDB ""
 rename-command DEBUG ""
  • 修改最大客户端连接数
# maxclients 10000
maxclients 50000
  • 修改最大内存限制
    如果是redis专用服务器,建议不超过物理内存的3/4
# maxmemory <bytes>
maxmemory 32GB
  • 保持淘汰策略不变-不淘汰
    通常情况下应该选择一个淘汰策略,而不是不淘汰
# maxmemory-policy noeviction
  • 打开异步操作,防止单线程阻塞
    • lazyfree-lazy-eviction:表示当 Redis 运行内存超过 maxmeory 时,是否开启 lazy free 机制删除;
    • lazyfree-lazy-expire:表示设置了过期时间的键值,当过期之后是否开启 lazy free 机制删除;
    • lazyfree-lazy-server-del:有些指令在处理已存在的键时,会带有一个隐式的 del 键的操作,比如 rename 命令,当目标键已存在,Redis 会先删除目标键,如果这些目标键是一个 big key,就会造成阻塞删除的问题,此配置表示在这种场景中是否开启 lazy free 机制删除;
    • slave-lazy-flush:针对 slave(从节点) 进行全量数据同步,slave 在加载 master 的 RDB 文件前,会运行 flushall 来清理自己的数据,它表示此时是否开启 lazy free 机制删除。
    • lazyfree-lazy-user-del:表示是否将 DEL 指令的默认行为替换成 lazy free 机制删除,效果就跟 UNLINK 一样。
############################# LAZY FREEING ####################################
lazyfree-lazy-eviction yes
lazyfree-lazy-expire yes
lazyfree-lazy-server-del yes
replica-lazy-flush yes
lazyfree-lazy-user-del yes
  • 修改慢查询日志设置
################################## SLOW LOG ###################################
# 修改为20000微秒,默认是10000微秒
slowlog-log-slower-than 20000
# 修改为保存1000条记录,默认128条
slowlog-max-len 1000

4.2.7 命令修改配置文件

  • 拷贝配置文件到安装路径下并修改。
cp ./redis.conf ${REDIS_HOME}/ && cd ${REDIS_HOME}
  • 先定义以下变量
    • PORT: Redis服务监听端口
    • MAXMEMORY:Redis服务最大内存限制
    • YPWD:Redis服务密码
cd ${REDIS_HOME}
PORT=6379
MAXMEMORY='32GB'
YPWD='yourpassword'
sed -i "s|bind 127.0.0.1 -::1| bind * -::* |" ./redis.conf \
&& sed -i "s|port 6379| port ${PORT}|" ./redis.conf \
&& sed -i "s|tcp-backlog 511| tcp-backlog 4096|" ./redis.conf \
&& sed -i "s|daemonize no| daemonize yes|" ./redis.conf \
&& sed -i "s|pidfile /var/run/redis_6379.pid| pidfile /var/run/redis/redis_${PORT}.pid|" ./redis.conf \
&& sed -i "s|logfile \"\"| logfile \"/var/log/redis/redis_${PORT}.log\"|" ./redis.conf \
&& sed -i "s|# save 3600 1 300 100 60 10000| save 3600 1 900 500 60 10000|" ./redis.conf \
&& sed -i "s|# save 3600 1| save 3600 1|" ./redis.conf \
&& sed -i "s|# save 300 100| save 900 500|" ./redis.conf \
&& sed -i "s|# save 60 10000| save 60 10000|" ./redis.conf \
&& sed -i "s|dbfilename dump.rdb| dbfilename dump_${PORT}.rdb|" ./redis.conf \
&& sed -i "s|dir ./| dir ${REDIS_HOME}/|" ./redis.conf \
&& sed -i "s|# requirepass foobared| requirepass ${YPWD}|" ./redis.conf \
&& sed -i "/# rename-command CONFIG \"\"/ a\\ rename-command CONFIG 'xkconfig'\n rename-command FLUSHALL ''\n rename-command FLUSHDB ''\n rename-command DEBUG ''\n"  ./redis.conf \
&& sed -i "s|# maxclients 10000| maxclients 50000|" ./redis.conf \
&& sed -i "s|# maxmemory <bytes>| maxmemory ${MAXMEMORY}|" ./redis.conf \
&& sed -i "s|lazyfree-lazy-eviction no| lazyfree-lazy-eviction yes|" ./redis.conf \
&& sed -i "s|lazyfree-lazy-expire no| lazyfree-lazy-expire yes|" ./redis.conf \
&& sed -i "s|lazyfree-lazy-server-del no| lazyfree-lazy-server-del yes|" ./redis.conf \
&& sed -i "s|replica-lazy-flush no| replica-lazy-flush yes|" ./redis.conf \
&& sed -i "s|lazyfree-lazy-user-del no| lazyfree-lazy-user-del yes|" ./redis.conf \
&& sed -i "s|slowlog-log-slower-than 10000| slowlog-log-slower-than 20000|" ./redis.conf \
&& sed -i "s|slowlog-max-len 128| slowlog-max-len 1000|" ./redis.conf \
  • 验证修改配置
cat ${REDIS_HOME}/redis.conf | grep -E "^[^;#]"

4.2.8 Redis使用目录赋权

# 日志目录赋权
chown -R redis:redis /var/log/redis
# 程序目录赋权
chown -R redis:redis /usr/local/redis
# pid文件目录赋权
chown -R redis:redis /var/run/redis
# 数据目录赋权(如果单独指定了数据目录),与配置文件中rdb存储路径一致
#chown -R redis:redis /data/redis/
chown -R redis:redis /var/log/redis \
&& chown -R redis:redis /var/run/redis \
&& chown -R redis:redis ${REDIS_HOME} \

4.2.9 启动Redis验证

  • 使用普通用户启动Redis
# sudo -u redis /usr/local/redis/bin/redis-server /usr/local/redis/redis.conf 
 sudo -u redis ${REDIS_HOME}/bin/redis-server ${REDIS_HOME}/redis.conf 
  • Redis已经监听6379端口
root@Virtual-Machine-203:~# netstat -ntlp| grep redis
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      19821/redis-server  
tcp6       0      0 ::1:6379                :::*                    LISTEN      19821/redis-server  
root@Virtual-Machine-203:~# redis-cli 
127.0.0.1:6379> auth yourpassword
OK
127.0.0.1:6379> select 0
OK
127.0.0.1:6379> dbsize
(integer) 0
127.0.0.1:6379> kkconfig get save
1) "save"
2) "3600 1 900 500 60 10000"
127.0.0.1:6379> 

4.2.10 目录结构

  1. Redis安装目录
  • ${REDIS_HOME} = /etc/usr/redis7.2.5
    • bin 程序目录,已添加到系统环境变量PATH中
    • dump_port.rdb 持久化rdb文件,每个对应一个节点
    • redis.conf 单例服务配置文件
[root@localhost redis7.2.5]# tree
.
├── bin
│   ├── redis-benchmark
│   ├── redis-check-aof -> redis-server
│   ├── redis-check-rdb -> redis-server
│   ├── redis-cli
│   ├── redis-sentinel -> redis-server
│   └── redis-server
├── dump_6379.rdb
└── redis.conf
  1. Redis日志目录
  • /var/log/redis
    • 每个Redis服务实例对应一个日志文件
[root@localhost redis]# tree
.
└── redis_6379.log
  1. Redis PID目录
  • /var/run/redis
    • 每个Redis服务实例对应一个PID文件
[root@localhost redis]# tree
.
└── redis_6379.pid

4.3 配置systemd管理

4.3.1 编译安装时未使用systemd

非正规做法!正常应该编译时指定使用systemd
https://github.com/redis/redis/commit/129d14e1431e913426485526663e1a9aac67838c

在这里插入图片描述

vim /etc/systemd/system/redis-server.service
# example systemd service unit file for redis-server
#
# In order to use this as a template for providing a redis service in your
# environment, _at the very least_ make sure to adapt the redis configuration
# file you intend to use as needed (make sure to set "supervised systemd"), and
# to set sane TimeoutStartSec and TimeoutStopSec property values in the unit's
# "[Service]" section to fit your needs.
#
# Some properties, such as User= and Group=, are highly desirable for virtually
# all deployments of redis, but cannot be provided in a manner that fits all
# expectable environments. Some of these properties have been commented out in
# this example service unit file, but you are highly encouraged to set them to
# fit your needs.
#
# Please refer to systemd.unit(5), systemd.service(5), and systemd.exec(5) for
# more information.

[Unit]
Description=Redis data structure server
Documentation=https://redis.io/documentation
#Before=your_application.service another_example_application.service
#AssertPathExists=/var/lib/redis
Wants=network-online.target
After=network-online.target

[Service]
# 配置正确的服务和配置文件路径
ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/redis.conf --supervised systemd --daemonize no
## Alternatively, have redis-server load a configuration file:
#ExecStart=/usr/local/bin/redis-server /path/to/your/redis.conf
ExecStop=/bin/kill -s TERM $MAINPID
LimitNOFILE=65535
NoNewPrivileges=yes
#OOMScoreAdjust=-900
PrivateTmp=yes
#Type=notify  # 默认 simple
TimeoutStartSec=infinity
TimeoutStopSec=infinity
UMask=0077
User=redis
Group=redis
WorkingDirectory=/usr/local/redis
RuntimeDirectory=redis

[Install]
WantedBy=multi-user.target
  • 替换配置文件中Redis路径
    如果使用默认路径/usr/local/redis,不用执行
# sed -i "s|/usr/local/redis|${REDIS_HOME}|g" /etc/systemd/system/redis-server.service 

4.3.2 编译安装时使用了systemd方式

vim /etc/systemd/system/redis-server.service
# example systemd service unit file for redis-server
#
# In order to use this as a template for providing a redis service in your
# environment, _at the very least_ make sure to adapt the redis configuration
# file you intend to use as needed (make sure to set "supervised systemd"), and
# to set sane TimeoutStartSec and TimeoutStopSec property values in the unit's
# "[Service]" section to fit your needs.
#
# Some properties, such as User= and Group=, are highly desirable for virtually
# all deployments of redis, but cannot be provided in a manner that fits all
# expectable environments. Some of these properties have been commented out in
# this example service unit file, but you are highly encouraged to set them to
# fit your needs.
#
# Please refer to systemd.unit(5), systemd.service(5), and systemd.exec(5) for
# more information.

[Unit]
Description=Redis data structure server
Documentation=https://redis.io/documentation
#Before=your_application.service another_example_application.service
#AssertPathExists=/var/lib/redis
Wants=network-online.target
After=network-online.target

[Service]
# 配置正确的服务和配置文件路径
ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/redis.conf --supervised systemd --daemonize no
## Alternatively, have redis-server load a configuration file:
#ExecStart=/usr/local/bin/redis-server /path/to/your/redis.conf
LimitNOFILE=65535
NoNewPrivileges=yes
#OOMScoreAdjust=-900
PrivateTmp=yes
Type=notify
#TimeoutStartSec=infinity
#TimeoutStopSec=infinity
TimeoutStartSec=180
TimeoutStopSec=180
UMask=0077
User=redis
Group=redis
WorkingDirectory=/usr/local/redis
RuntimeDirectory=redis

[Install]
WantedBy=multi-user.target
  • 替换配置文件中Redis路径
    如果使用默认路径/usr/local/redis,不用执行
# sed -i "s|/usr/local/redis|${REDIS_HOME}|g" /etc/systemd/system/redis-server.service 

4.3.3 加载并验证服务

  • 加载systemd配置文件,每次修改后都需要重新加载
systemctl daemon-reload
  • 验证服务启动
# 启动
systemctl start redis-server
# 查看进程
netstat -ntlp | grep redis
  • 验证服务重启
# 测试重启
systemctl restart redis-server
# 查看进程ID是否变更
netstat -ntlp | grep redis
  • 验证服务停止
# 测试重启
systemctl stop redis-server
# 查看进程ID是否变更
netstat -ntlp | grep redis

4.3 部署常见问题

4.3.1 编译时环境问题

  • 现象:Ubuntu 没装 GCC
make[1]: Entering directory '/root/redis-7.2.5/src'
sh: 1: cc: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
    CC Makefile.dep
sh: 1: cc: not found
make[3]: 离开目录“/public/redis-7.2.5/deps/jemalloc”
make[2]: 离开目录“/public/redis-7.2.5/deps”
    CC adlist.o
    CC quicklist.o
    CC ae.o
    CC anet.o
    CC dict.o
    CC server.o
In file included from server.c:30:0:
server.h:58:31: 致命错误:systemd/sd-daemon.h:没有那个文件或目录
 #include <systemd/sd-daemon.h>
                               ^
编译中断。
make[1]: *** [server.o] 错误 1
make[1]: 离开目录“/public/redis-7.2.5/src”
make: *** [install] 错误 2
  • 处理:Ubuntu 22.04 安装
apt install build-essential pkg-config -y
  • 处理:如果是CentOS 7.9,参考安装
yum install make automake gcc gcc-c++ kernel-devel systemd-devel -y

4.3.2 编译时缺少文件

  • 现象:编译时缺文件
/usr/bin/ld: cannot find ../deps/hiredis/libhiredis.a: No such file or directory
/usr/bin/ld: cannot find ../deps/lua/src/liblua.a: No such file or directory
/usr/bin/ld: cannot find ../deps/hdr_histogram/libhdrhistogram.a: No such file or directory
/usr/bin/ld: cannot find ../deps/fpconv/libfpconv.a: No such file or directory
/usr/bin/ld: cannot find ../deps/jemalloc/lib/libjemalloc.a: No such file or directory
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:403: redis-server] Error 1
make[1]: Leaving directory '/root/redis-7.2.5/src'
make: *** [Makefile:9: install] Error 2
  • 处理:先编译缺少的文件
root@Virtual-Machine-203:~# cd deps/
root@Virtual-Machine-203:~# make hiredis lua hdr_histogram fpconv jemalloc linenoise

# 如果jemalloc仍然不行,可以试试以下方式
#root@Virtual-Machine-203:~# cd  deps/jemalloc/
#root@Virtual-Machine-203:~#  ./configure
#root@Virtual-Machine-203:~#  make && make install_bin install_include install_lib
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值