14 服务器
参考大佬代码：https://www.cnblogs.com/dpains/p/7640528.html
14.1 命令请求执行的过程
14.1.1 发送命令请求
- 用户在客户端输入命令请求，客户端会将这个命令请求转换成协议格式，然后通过连接到服务器的套接字，将协议格式的命令请求发送给服务器
14.1.2 读取命令请求
- 客户端与服务器之间的连接套接字因为客户端的写入而变得可读，服务器将调用命令请求处理器来执行一下操作
- 读取套接字协议格式的命令请求，并将其保存到客户端状态的输入缓冲区中
- 对输入缓冲区中的命令请求进行分析，提取出命令请求中包含的命令参数，以及命令参数的个数，将其保存到argv和argc里面
- 调用命令执行器，执行客户端指定的命令
14.1.3 命令执行器（1）：查找命令实现
- 查找命令实现
- 根据客户端argv[0],在命令表中查找参数指定的命令，并把命令保存到客户端状态的cmd属性中
- redisCommand
- name：命令的名字
- proc：函数指针，指向命令的实现函数，比如setCommand。redisCommandProc类型的定义为typedef void redisCommandProc
- arity：命令参数的个数，用于检查命令请求的格式是否正确。如果值为负数-N，表示参数的数量大于等于N
- sflags：字符串形式的标识值，记录了命令的属性，读命令还是写命令，是否允许在载入数据时使用，这个命令是否允许在Lua脚本中使用
- flags：对sflags标识进行分析得出的二进制标识，程序自动生成的。服务器对命令标识进行检查都用flags而不是sflags属性，因此对二进制标识的检查可以方便地通过&、^、～等操作完成
- calls：服务器总共执行了多少次这个命令
- milliseconds：服务器执行这个命令所耗费的总时长
//code0 server.h
struct redisCommand {
char *name;
redisCommandProc *proc;
int arity;
char *sflags; /* Flags as string representation, one char per flag. */
uint64_t flags; /* The actual flags, obtained from the 'sflags' field. */
/* Use a function to determine keys arguments in a command line.
* Used for Redis Cluster redirect. */
redisGetKeysProc *getkeys_proc;
/* What keys should be loaded in background when calling this command? */
int firstkey; /* The first argument that's a key (0 = no keys) */
int lastkey; /* The last argument that's a key */
int keystep; /* The step between first and last key */
long long microseconds, calls;
int id; /* Command ID. This is a progressive ID starting from 0 that
is assigned at runtime, and is used in order to check
ACLs. A connection is able to execute a given command if
the user associated to the connection has this command
bit set in the bitmap of allowed commands. */
};
- sflags的标识
- w：写入命令，可能会修改数据库
- r：只读命令，不会修改数据库
- m：可能会占用大量内存，在使用之前要检查服务器的内存使用情况，如果内存紧缺就禁止执行这个命令
- a：这是一个管理命令
- p：发布与订阅功能方面的命令
- s：这个命令不能在Lua脚本中使用
- R：随机命令，对于相同的数据集和相同的参数，命令返回的结果可能不同
- S：当在Lua脚本中使用这个命令时，对这个命令的输出结果进行一次排序，使得命令的结果有序
- l：命令在服务器载入数据的过程中使用
- t：允许从服务器在带有过期数据时使用的命令
- M：这个命令在监视器模式下不会自动被传播
- example
- SET命令名称为set，实现的函数为setCommand，命令的参数个数为-3，表示接受三个或者以上数量的参数，命令的标识为wm（写入命令，写入之前要进行内存检查，因为有可能会占用大量内存）
14.1.4 命令执行器（2）：执行预备操作（单机模式）
- 拿到了执行命令所需的一些命令实现函数、参数等，但是真正执行之前，需要检查
- cmd指针是否为NULL，如果为NULL，说明用户输入的命令名字找不到相应的命令实现，服务器不再执行后续步骤
- 检查命令参数个数是否符合arity要求。如果不符合，服务器向客户端返回一个错误
- 检查客户端是否通过了身份验证，没有通过身份验证的客户端只能执行auth；如果没有通过身份验证的客户端试图执行除了AUTH命令之外的其他命令，那么服务器将向客户端返回一个错误
- 服务器打开maxmemory功能，那么执行命令之前，先检查服务器的内存占用情况，并在有需要时进行内存回收，从而使得接下来的命令可以顺利执行。如果内存回收失败，则不再继续往后执行
- 服务器上一次执行BGSAVE命令出错，并且服务器打开了stop-writes-on-bgsave-error功能，而且服务器即将执行的命令是一个写命令，那么服务器将拒绝执行这个命令，并向客户端返回错误
- 如果客户端正在用SUBSCRIBE订阅频道，或者正在用PSUBSCRIBE命令订阅模式，那么服务器只会执行客户端发来的SUBSCRIBE、PSUBSCRIBE、UNSUBSCRIBE、PUNSUBSCRIBE四个命令，其余都会被服务器拒绝
- 如果服务器正在进行数据载入，那么客户端发送的命令必须带有l标识（比如INFO、SHUTDOWN、PUBLISH等）才会被服务器执行，其他都会被拒绝
- 如果服务器因为执行Lua脚本而超时并进入阻塞状态，那么服务器只会执行客户端发来的SHUTDOWN nosave命令和SCRIPT KILL命令，其余命令都会被服务器拒绝
- 如果客户端正在执行事务，那么服务器只会执行客户端发来的EXEC、DIACARD、MULTI、WATCH四个命令，其他命令都会被放进事务队列中
- 如果服务器打开了监视器功能，那么服务器将要执行的命令和参数会发给监视器。
14.1.5 命令执行器（3）：调用命令的实现函数
- client找到redisCommand，然后结合argc和argv调用redisCommandProc，回复会保存在client里面的buf和reply中
14.1.6 命令执行器（4）：后续操作
- 慢查询日志功能：如果服务器开启了慢查询日志功能，那么慢查询日志模块会检查是否需要给刚刚执行完的命令请求添加一条新的慢查询日志
- 更新时长和执行次数：更新redisCommand结构的milliseconds属性，同时将命令的redisCommand结构的calls计数器+1
- AOF持久化功能：如果服务器开启了AOF持久化功能，AOF持久化某块会将刚刚执行的命令写入AOF缓冲区中
- 如果其他从服务器正在复制当前这个服务器，这个服务器会将刚刚执行的命令传播给所有的从服务器
14.1.7 命令回复发给客户端
- 命令回复保存到客户端的输出缓冲区里面，并为客户端的套接字关联命令回复处理器
- 当客户端套接字变成可写状态时，服务器就会执行命令回复处理器，将保存在客户端输出缓冲区的命令回复发给客户端
- 命令回复发送完毕，回复处理器会清空客户端状态的输出缓冲区，为处理下一个命令请求做好准备
14.1.8 客户端接受并打印命令回复
- 客户端接收到协议格式的命令回复之后，会将这些回复转换成人类可读的格式，然后打印
14.2 serverCron函数
- 默认每隔100ms执行一次，负责管理服务器的资源并保持服务器自身的良好运转
14.2.1 更新服务器时间缓存
- redis服务器很多功能需要获取系统的当前时间，但每次获取系统的当前时间都需要执行一次系统调用。为了减少系统调用的执行次数，服务器状态的unixtime属性和mstime属性被用作当前时间的缓存。
- serverCron默认会每100ms更新unixtime和mstime，所以这两个属性记录的时间精度不高
- 对时间精度要求不高的操作使用unixtime和mstime属性：服务器打印日志、更新服务器的LRU时钟、决定是否执行持久化任务、计算服务器上线时间等
- 对时间精度要求比较高：为键设置过期时间、添加慢查询日志等
//code5: server.c
void updateCachedTime(int update_daylight_info) {
server.ustime = ustime();
server.mstime = server.ustime / 1000;
server.unixtime = server.mstime / 1000;
/* To get information about daylight saving time, we need to call
* localtime_r and cache the result. However calling localtime_r in this
* context is safe since we will never fork() while here, in the main
* thread. The logging function will call a thread safe version of
* localtime that has no locks. */
if (update_daylight_info) {
struct tm tm;
time_t ut = server.unixtime;
localtime_r(&ut,&tm);
server.daylight_active = tm.tm_isdst;
}
}
//code1: server.h
struct redisServer {
/* General */
// ...
_Atomic unsigned int lruclock; /* Clock for LRU eviction */
_Atomic time_t unixtime; /* Unix time sampled every cron cycle. */
time_t timezone; /* Cached timezone. As set by tzset(). */
int daylight_active; /* Currently in daylight saving time. */
mstime_t mstime; /* 'unixtime' in milliseconds. */ 毫秒
ustime_t ustime; /* 'unixtime' in microseconds. */ 微秒，百万分之一秒
// ...
}
14.2.2 更新LRU时钟
- 服务器状态中的lruclock保存了服务器的LRU时钟（serverCron函数默认每10s更新lrurock属性的值）
- 每个Redis对象都有一个lru属性，记录该对象最后一次被命令访问的时间
- 服务器要计算一个数据库键的空转时间，会使用lrurock属性记录的时间去减去对象的lru属性记录的时间，得出的计算结果就是这个对象的空转时间
//code2: server.h
typedef struct redisObject {
unsigned type:4;
unsigned encoding:4;
unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
* LFU data (least significant 8 bits frequency
* and most significant 16 bits access time). */
int refcount;
void *ptr;
} robj;
14.2.3 更新服务器每秒执行命令次数
- 查看服务器在最近一秒钟处理的命令请求数量，使用info status查看instantaneous_ops_per_sec
- trackInstantaneousMetrice函数每100毫秒运行一次
- getInstantaneousMetric得到的是一个平均值
//code3: server.h
struct redisServer {
/* General */
// ...
/* The following two are used to track instantaneous metrics, like
* number of operations per second, network traffic. */
struct {
long long last_sample_time; /* Timestamp of last sample in ms */
long long last_sample_count;/* Count in last sample */
long long samples[STATS_METRIC_SAMPLES];
int idx;
} inst_metric[STATS_METRIC_COUNT];
size_t stat_peak_memory; /* Max used memory record */
// ...
//code4: server.c
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
//...
run_with_period(100) {
trackInstantaneousMetric(STATS_METRIC_COMMAND,server.stat_numcommands);
trackInstantaneousMetric(STATS_METRIC_NET_INPUT,
server.stat_net_input_bytes);
trackInstantaneousMetric(STATS_METRIC_NET_OUTPUT,
server.stat_net_output_bytes);
}
//...
}
/* ======================= Cron: called every 100 ms ======================== */
/* Add a sample to the operations per second array of samples. */
void trackInstantaneousMetric(int metric, long long current_reading) {
long long t = mstime() - server.inst_metric[metric].last_sample_time;
long long ops = current_reading -
server.inst_metric[metric].last_sample_count;
long long ops_sec;
ops_sec = t > 0 ? (ops*1000/t) : 0;
server.inst_metric[metric].samples[server.inst_metric[metric].idx] =
ops_sec;
server.inst_metric[metric].idx++;
server.inst_metric[metric].idx %= STATS_METRIC_SAMPLES;
server.inst_metric[metric].last_sample_time = mstime();
server.inst_metric[metric].last_sample_count = current_reading;
}
/* Return the mean of all the samples. */
long long getInstantaneousMetric(int metric) {
int j;
long long sum = 0;
for (j = 0; j < STATS_METRIC_SAMPLES; j++)
sum += server.inst_metric[metric].samples[j];
return sum / STATS_METRIC_SAMPLES;
}
14.2.4 更新服务器内存峰值记录
- 如果当前使用的内存数量比stat_peak_memory属性记录的值要大，则更新
//code6: server.h
struct redisServer {
/* General */
// ...
size_t stat_peak_memory; /* Max used memory record */
// ...
}
//server.c
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
//...
if (zmalloc_used_memory() > server.stat_peak_memory)
server.stat_peak_memory = zmalloc_used_memory();
//...
}
14.2.5 处理SIGTERM信号
- 启动服务器的时候，Redis会为服务器进行的SIGTERM信号关联sigShutdownHandler函数，函数内部会打开服务器状态的shutdown_asap标识
- serverCron函数运行时，程序会判断shutdown_asap属性的值，根据值判断是否关闭服务器（服务器关闭自身之前会进行RDB持久化操作）
//code7: server.c
static void sigShutdownHandler(int sig) {
char *msg;
switch (sig) {
case SIGINT:
msg = "Received SIGINT scheduling shutdown...";
break;
case SIGTERM:
msg = "Received SIGTERM scheduling shutdown...";
break;
default:
msg = "Received shutdown signal, scheduling shutdown...";
};
/* SIGINT is often delivered via Ctrl+C in an interactive session.
* If we receive the signal the second time, we interpret this as
* the user really wanting to quit ASAP without waiting to persist
* on disk. */
if (server.shutdown_asap && sig == SIGINT) {
serverLogFromHandler(LL_WARNING, "You insist... exiting now.");
rdbRemoveTempFile(getpid());
exit(1); /* Exit with an error since this was not a clean shutdown. */
} else if (server.loading) {
serverLogFromHandler(LL_WARNING, "Received shutdown signal during loading, exiting now.");
exit(0);
}
serverLogFromHandler(LL_WARNING, msg);
server.shutdown_asap = 1;
}
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
//...
/* We received a SIGTERM, shutting down here in a safe way, as it is
* not ok doing so inside the signal handler. */
if (server.shutdown_asap) {
if (prepareForShutdown(SHUTDOWN_NOFLAGS) == C_OK) exit(0);
serverLog(LL_WARNING,"SIGTERM received but errors trying to shut down the server, check the logs for more information");
server.shutdown_asap = 0;
}
//...
}
14.2.6 管理客户端资源
- 会进行两个检查
- 客户端与服务器的连接已经超时，很长一段时间里面客户端和服务器都没有互动，那么程序释放这个客户端
- 客户端在上一次执行完命令请求后，输入缓冲区的大小超过了一定的长度，那么程序会释放客户端当前的输入缓冲区，并重新创建一个默认大小的输入缓冲区，防止客户端的输入缓冲区耗费了过多的内存
//code8: server.c
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
//...
/* We need to do a few operations on clients asynchronously. */
clientsCron();
/* Handle background operations on Redis databases. */
databasesCron();
//...
}
void clientsCron(void) {
/* Try to process at least numclients/server.hz of clients
* per call. Since normally (if there are no big latency events) this
* function is called server.hz times per second, in the average case we
* process all the clients in 1 second. */
int numclients = listLength(server.clients);
int iterations = numclients/server.hz;
mstime_t now = mstime();
/* Process at least a few clients while we are at it, even if we need
* to process less than CLIENTS_CRON_MIN_ITERATIONS to meet our contract
* of processing each client once per second. */
if (iterations < CLIENTS_CRON_MIN_ITERATIONS)
iterations = (numclients < CLIENTS_CRON_MIN_ITERATIONS) ?
numclients : CLIENTS_CRON_MIN_ITERATIONS;
while(listLength(server.clients) && iterations--) {
client *c;
listNode *head;
/* Rotate the list, take the current head, process.
* This way if the client must be removed from the list it's the
* first element and we don't incur into O(N) computation. */
listRotateTailToHead(server.clients);
head = listFirst(server.clients);
c = listNodeValue(head);
/* The following functions do different service checks on the client.
* The protocol is that they return non-zero if the client was
* terminated. */
if (clientsCronHandleTimeout(c,now)) continue;
if (clientsCronResizeQueryBuffer(c)) continue;
if (clientsCronTrackExpansiveClients(c)) continue;
if (clientsCronTrackClientsMemUsage(c)) continue;
}
}
14.2.7 管理数据库资源
- 删除过期键
- 有需要，会对字典进行收缩
//code9 server.c
void databasesCron(void) {
/* Expire keys by random sampling. Not required for slaves
* as master will synthesize DELs for us. */
if (server.active_expire_enabled) {
if (iAmMaster()) {
activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);
} else {
expireSlaveKeys();
}
}
/* Defrag keys gradually. */
activeDefragCycle();
/* Perform hash tables rehashing if needed, but only if there are no
* other processes saving the DB on disk. Otherwise rehashing is bad
* as will cause a lot of copy-on-write of memory pages. */
if (!hasActiveChildProcess()) {
/* We use global counters so if we stop the computation at a given
* DB we'll be able to start from the successive in the next
* cron loop iteration. */
static unsigned int resize_db = 0;
static unsigned int rehash_db = 0;
int dbs_per_call = CRON_DBS_PER_CALL;
int j;
/* Don't test more DBs than we have. */
if (dbs_per_call > server.dbnum) dbs_per_call = server.dbnum;
/* Resize */
for (j = 0; j < dbs_per_call; j++) {
tryResizeHashTables(resize_db % server.dbnum);
resize_db++;
}
/* Rehash */
if (server.activerehashing) {
for (j = 0; j < dbs_per_call; j++) {
int work_done = incrementallyRehash(rehash_db);
if (work_done) {
/* If the function did some work, stop here, we'll do
* more at the next cron loop. */
break;
} else {
/* If this db didn't need rehash, we'll try the next one. */
rehash_db++;
rehash_db %= server.dbnum;
}
}
}
}
}
14.2.8 执行被延迟的BGREWRITEANOF
- 服务器执行BGSAVE命令，如果客户端向服务器发来BGREWRITEANOF命令，服务器会将BGREWRITEANOF命令延迟到BGSAVE执行完之后，用服务器状态里面的aof_rewrite_scheduled变量记录服务器是否延迟了BGREWRITEANOF命令
- 每次serverCron函数执行的时候，函数会检查BGSAVE和BGREWRITEAOF是否在执行，如果不在执行，且aof_rewrite_scheduled=1，那么服务器就会执行之前被推延的BGREWRITEAOF命令
//code10: server.h
struct redisServer {
/* General */
// ...
int aof_rewrite_scheduled; /* Rewrite once BGSAVE terminates. */
// ...
}
//server.c
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
//...
/* Start a scheduled AOF rewrite if this was requested by the user while
* a BGSAVE was in progress. */
if (!hasActiveChildProcess() &&
server.aof_rewrite_scheduled)
{
rewriteAppendOnlyFileBackground();
}
//...
}
14.2.9 检查持久化操作的运行状态
- 服务器状态使用rdb_child_pid属性和aof_child_pid属性记录BGSAVE和BGREWRITEAOF命令的子进程id，这两个属性也可以用于检查BGSAVE或者BGREWRITEAOF命令是否正在执行
- rdb_child_pid: 记录BGSAVE命令的子进程ID，如果服务器没有在执行BGSAVE，属性值为-1
- aof_child_pid：记录BGREWRITEAOF的子进程ID，如果服务器没有在执行BGREWRITEAOF，属性值为-1
- serverCron函数会检查这两个值是否为-1，如果有一个不为-1，程序会调用checkChildrenDone函数，该函数中会执行一次wait3函数，检查子进程是否有信号发来服务器进程
- 如果有信号到达，表示新的RDB文件已经生成完毕或者AOF文件已经重写完毕，服务器需要进行相应命令的后续操作（用新的RDB文件替换现有的RDB文件，或者用重写后的AOF文件替换现有的AOF文件）
- 如果信号没有到达，证明持久化操作还未完成，程序不做动作
- serverCron函数会检查这两个值是否为-1，如果都为-1，表明服务器没有在进行持久化操作。此情况下，程序需执行三个检查（serverCron函数里的代码很明显）
- 是否有BGREWRITEAOF被延迟，如果有，开始一次新的BGREWRITEAOF操作
- 检查服务器的自动保存条件是否已经被满足，如果满足且服务器没有在执行其他的持久化操作，那么服务器开始一次新的BGSAVE操作
- 检查服务器设置的AOF重写条件是否满足，如果条件满足，并且服务器没有在执行其他持久化操作，那么服务器将开始一次新的BGREWRITEAOF操作
//code11: server.h
struct redisServer {
/* General */
// ...
pid_t rdb_child_pid; /* PID of RDB saving child */
pid_t aof_child_pid; /* PID if rewriting process */
// ...
}
//server.c
/* Return true if there are no active children processes doing RDB saving,
* AOF rewriting, or some side process spawned by a loaded module. */
int hasActiveChildProcess() {
return server.rdb_child_pid != -1 ||
server.aof_child_pid != -1 ||
server.module_child_pid != -1;
}
void checkChildrenDone(void) {
int statloc;
pid_t pid;
/* If we have a diskless rdb child (note that we support only one concurrent
* child), we want to avoid collecting it's exit status and acting on it
* as long as we didn't finish to drain the pipe, since then we're at risk
* of starting a new fork and a new pipe before we're done with the previous
* one. */
if (server.rdb_child_pid != -1 && server.rdb_pipe_conns)
return;
if ((pid = wait3(&statloc,WNOHANG,NULL)) != 0) {
int exitcode = WEXITSTATUS(statloc);
int bysignal = 0;
if (WIFSIGNALED(statloc)) bysignal = WTERMSIG(statloc);
/* sigKillChildHandler catches the signal and calls exit(), but we
* must make sure not to flag lastbgsave_status, etc incorrectly.
* We could directly terminate the child process via SIGUSR1
* without handling it, but in this case Valgrind will log an
* annoying error. */
if (exitcode == SERVER_CHILD_NOERROR_RETVAL) {
bysignal = SIGUSR1;
exitcode = 1;
}
if (pid == -1) {
serverLog(LL_WARNING,"wait3() returned an error: %s. "
"rdb_child_pid = %d, aof_child_pid = %d, module_child_pid = %d",
strerror(errno),
(int) server.rdb_child_pid,
(int) server.aof_child_pid,
(int) server.module_child_pid);
} else if (pid == server.rdb_child_pid) {
backgroundSaveDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else if (pid == server.aof_child_pid) {
backgroundRewriteDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else if (pid == server.module_child_pid) {
ModuleForkDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else {
if (!ldbRemoveChild(pid)) {
serverLog(LL_WARNING,
"Warning, detected child with unmatched pid: %ld",
(long)pid);
}
}
updateDictResizePolicy();
closeChildInfoPipe();
}
}
14.2.10 检查持久化操作的运行状态
- 服务器开启了AOF持久化功能，而且AOF缓冲区里面还有待写入的数据，那么serverCron会调用相应的程序，将AOF缓冲区的内容写入到AOF文件（调用函数：rewriteAppendOnlyFileBackground()）
//code12: server.c
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
//...
if (hasActiveChildProcess() || ldbPendingChildren())
{
checkChildrenDone();
} else {
/* If there is not a background saving/rewrite in progress check if
* we have to save/rewrite now. */
for (j = 0; j < server.saveparamslen; j++) {
struct saveparam *sp = server.saveparams+j;
/* Save if we reached the given amount of changes,
* the given amount of seconds, and if the latest bgsave was
* successful or if, in case of an error, at least
* CONFIG_BGSAVE_RETRY_DELAY seconds already elapsed. */
if (server.dirty >= sp->changes &&
server.unixtime-server.lastsave > sp->seconds &&
(server.unixtime-server.lastbgsave_try >
CONFIG_BGSAVE_RETRY_DELAY ||
server.lastbgsave_status == C_OK))
{
serverLog(LL_NOTICE,"%d changes in %d seconds. Saving...",
sp->changes, (int)sp->seconds);
rdbSaveInfo rsi, *rsiptr;
rsiptr = rdbPopulateSaveInfo(&rsi);
rdbSaveBackground(server.rdb_filename,rsiptr);
break;
}
}
/* Trigger an AOF rewrite if needed. */
if (server.aof_state == AOF_ON &&
!hasActiveChildProcess() &&
server.aof_rewrite_perc &&
server.aof_current_size > server.aof_rewrite_min_size)
{
long long base = server.aof_rewrite_base_size ?
server.aof_rewrite_base_size : 1;
long long growth = (server.aof_current_size*100/base) - 100;
if (growth >= server.aof_rewrite_perc) {
serverLog(LL_NOTICE,"Starting automatic rewriting of AOF on %lld%% growth",growth);
rewriteAppendOnlyFileBackground();
}
}
}
//...
}
14.2.11 关闭异步客户端
- 服务器会关闭那些输出缓冲区大小超出限制的客户端
14.2.12 增加cronloops计数器的值
- serverCron函数中会执行server.cronloops++;
- 作用：在一些别的复制模块中，serverCron每执行N次就执行一次指定代码
14.3 服务器的启动过程（初始化）
14.3.1 初始化服务器状态结构
- 设置服务器的运行ID：getRandomHexChars
- 设置服务器的默认运行频率
- 设置服务器的默认配置文件路径
- 设置服务器的默认端口号
- 设置服务器的默认RDB和AOF持久化条件
- 初始化服务器的LRU时钟
- 创建命令表：populateCommandTable()
//code13：server.c
void initServerConfig(void) {
int j;
updateCachedTime(1);
// 设置服务器的运行ID
getRandomHexChars(server.runid,CONFIG_RUN_ID_SIZE);
server.runid[CONFIG_RUN_ID_SIZE] = '\0';
changeReplicationId();
clearReplicationId2();
server.hz = CONFIG_DEFAULT_HZ; /* Initialize it ASAP, even if it may get
updated later after loading the config.
This value may be used before the server
is initialized. */
server.timezone = getTimeZone(); /* Initialized by tzset(). */
server.configfile = NULL;
server.executable = NULL;
server.arch_bits = (sizeof(long) == 8) ? 64 : 32;
server.bindaddr_count = 0;
server.unixsocketperm = CONFIG_DEFAULT_UNIX_SOCKET_PERM;
server.ipfd_count = 0;
server.tlsfd_count = 0;
server.sofd = -1;
server.active_expire_enabled = 1;
server.client_max_querybuf_len = PROTO_MAX_QUERYBUF_LEN;
server.saveparams = NULL;
server.loading = 0;
server.logfile = zstrdup(CONFIG_DEFAULT_LOGFILE);
server.aof_state = AOF_OFF;
server.aof_rewrite_base_size = 0;
server.aof_rewrite_scheduled = 0;
server.aof_flush_sleep = 0;
server.aof_last_fsync = time(NULL);
server.aof_rewrite_time_last = -1;
server.aof_rewrite_time_start = -1;
server.aof_lastbgrewrite_status = C_OK;
server.aof_delayed_fsync = 0;
server.aof_fd = -1;
server.aof_selected_db = -1; /* Make sure the first time will not match */
server.aof_flush_postponed_start = 0;
server.pidfile = NULL;
server.active_defrag_running = 0;
server.notify_keyspace_events = 0;
server.blocked_clients = 0;
memset(server.blocked_clients_by_type,0,
sizeof(server.blocked_clients_by_type));
server.shutdown_asap = 0;
server.cluster_configfile = zstrdup(CONFIG_DEFAULT_CLUSTER_CONFIG_FILE);
server.cluster_module_flags = CLUSTER_MODULE_FLAG_NONE;
server.migrate_cached_sockets = dictCreate(&migrateCacheDictType,NULL);
server.next_client_id = 1; /* Client IDs, start from 1 .*/
server.loading_process_events_interval_bytes = (1024*1024*2);
server.lruclock = getLRUClock();
resetServerSaveParams();
appendServerSaveParams(60*60,1); /* save after 1 hour and 1 change */
appendServerSaveParams(300,100); /* save after 5 minutes and 100 changes */
appendServerSaveParams(60,10000); /* save after 1 minute and 10000 changes */
/* Replication related */
server.masterauth = NULL;
server.masterhost = NULL;
server.masterport = 6379;
server.master = NULL;
server.cached_master = NULL;
server.master_initial_offset = -1;
server.repl_state = REPL_STATE_NONE;
server.repl_transfer_tmpfile = NULL;
server.repl_transfer_fd = -1;
server.repl_transfer_s = NULL;
server.repl_syncio_timeout = CONFIG_REPL_SYNCIO_TIMEOUT;
server.repl_down_since = 0; /* Never connected, repl is down since EVER. */
server.master_repl_offset = 0;
/* Replication partial resync backlog */
server.repl_backlog = NULL;
server.repl_backlog_histlen = 0;
server.repl_backlog_idx = 0;
server.repl_backlog_off = 0;
server.repl_no_slaves_since = time(NULL);
/* Client output buffer limits */
for (j = 0; j < CLIENT_TYPE_OBUF_COUNT; j++)
server.client_obuf_limits[j] = clientBufferLimitsDefaults[j];
/* Linux OOM Score config */
for (j = 0; j < CONFIG_OOM_COUNT; j++)
server.oom_score_adj_values[j] = configOOMScoreAdjValuesDefaults[j];
/* Double constants initialization */
R_Zero = 0.0;
R_PosInf = 1.0/R_Zero;
R_NegInf = -1.0/R_Zero;
R_Nan = R_Zero/R_Zero;
/* Command table -- we initiialize it here as it is part of the
* initial configuration, since command names may be changed via
* redis.conf using the rename-command directive. */
server.commands = dictCreate(&commandTableDictType,NULL);
server.orig_commands = dictCreate(&commandTableDictType,NULL);
populateCommandTable();
server.delCommand = lookupCommandByCString("del");
server.multiCommand = lookupCommandByCString("multi");
server.lpushCommand = lookupCommandByCString("lpush");
server.lpopCommand = lookupCommandByCString("lpop");
server.rpopCommand = lookupCommandByCString("rpop");
server.zpopminCommand = lookupCommandByCString("zpopmin");
server.zpopmaxCommand = lookupCommandByCString("zpopmax");
server.sremCommand = lookupCommandByCString("srem");
server.execCommand = lookupCommandByCString("exec");
server.expireCommand = lookupCommandByCString("expire");
server.pexpireCommand = lookupCommandByCString("pexpire");
server.xclaimCommand = lookupCommandByCString("xclaim");
server.xgroupCommand = lookupCommandByCString("xgroup");
server.rpoplpushCommand = lookupCommandByCString("rpoplpush");
/* Debugging */
server.assert_failed = "<no assertion failed>";
server.assert_file = "<no file>";
server.assert_line = 0;
server.bug_report_start = 0;
server.watchdog_period = 0;
/* By default we want scripts to be always replicated by effects
* (single commands executed by the script), and not by sending the
* script to the slave / AOF. This is the new way starting from
* Redis 5. However it is possible to revert it via redis.conf. */
server.lua_always_replicate_commands = 1;
initConfigValues();
}
14.3.2 接受用户指定的服务器配置
- 启动服务器时，用户可以通过给定配置参数或者指定配置文件来修改服务器的默认配置
- initServerConfig函数的最后一行会调用initConfigValues，根据用户的配置，对server变量相关属性的值进行修改
$ redis-server --port 10086
$ redis-server redis.conf
//code14: config.c
void initConfigValues() {
for (standardConfig *config = configs; config->name != NULL; config++) {
config->interface.init(config->data);
}
}
14.3.3 创建相应的数据结构
- initServerConfig函数中，只创建了命令表一个数据结构，除了命令表之外，服务器还包含其他数据结构
- server.clients链表，包含所有与服务器相连的客户端的状态结构
- server.db，包含了服务器的所有数据库
- 保存频道订阅信息的server.pubpub_channels字典；保存模式订阅信息的server.pubpub_patterns链表
- server.lua，用于执行Lua脚本的Lua环境
- server.slowlog，用于保存慢查询日志
- 服务器必须先载入用户指定的配置选项，才能正确地对数据结构进行初始化。initServerConfig先初始化一般属性，而initServer负责初始化数据结构
- 其他设置
- 服务器设置进程信号处理器（C++ signal）
- SIGHUP: hangup detected on controlling terminal or death of controlling process
- SIGPIPE: broken pipe, write to pipe with no readers
- 创建共享对象
- 打开服务器的监听端口，并为监听套接字关联连接应答事件处理器，等待服务器正式运行时接受客户端的连接
- 为serverCron创建时间事件，等待服务器正式运行时执行serverCron函数
- AOF持久化功能已经打开，那么打开现有的AOF文件。如果AOF不存在，则创建并打开一个新的AOF文件，为AOF写入做好准备
- 书里此处提到了初始化服务器的后台I/O模块（bioInit），发现在server.c的main函数里面执行了
- 服务器设置进程信号处理器（C++ signal）
$ redis-server --port 10086
$ redis-server redis.conf
//code15: server.c
void initServer(void) {
int j;
signal(SIGHUP, SIG_IGN);
signal(SIGPIPE, SIG_IGN);
setupSignalHandlers();
//...
createSharedObjects();
//...
/* Open the TCP listening socket for the user commands. */
if (server.port != 0 &&
listenToPort(server.port,server.ipfd,&server.ipfd_count) == C_ERR)
exit(1);
if (server.tls_port != 0 &&
listenToPort(server.tls_port,server.tlsfd,&server.tlsfd_count) == C_ERR)
exit(1);
//...
/* Create the timer callback, this is our way to process many background
* operations incrementally, like clients timeout, eviction of unaccessed
* expired keys and so forth. */
if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) {
serverPanic("Can't create event loop timers.");
exit(1);
}
//...
/* Open the AOF file if needed. */
if (server.aof_state == AOF_ON) {
server.aof_fd = open(server.aof_filename,
O_WRONLY|O_APPEND|O_CREAT,0644);
if (server.aof_fd == -1) {
serverLog(LL_WARNING, "Can't open the append-only file: %s",
strerror(errno));
exit(1);
}
}
//...
}
// exec in main func
/* Some steps in server initialization need to be done last (after modules
* are loaded).
* Specifically, creation of threads due to a race bug in ld.so, in which
* Thread Local Storage initialization collides with dlopen call.
* see: https://sourceware.org/bugzilla/show_bug.cgi?id=19329 */
void InitServerLast() {
bioInit();
initThreadedIO();
set_jemalloc_bg_thread(server.jemalloc_bg_thread);
server.initial_memory_usage = zmalloc_used_memory();
}
14.3.4 还原数据库状态
- 完成对服务器状态server变量的初始化之后，服务器需要载入rdb文件或者aof文件
- 检查是否启用了AOF持久化，启用了，服务器使用AOF文件还原数据库状态
- 没有启动AOF持久化，服务器使用RDB文件来还原数据库状态
- 服务器完成数据库状态还原工作之后，服务器将在日志中打印载入文件还原数据库耗费的时长
14.3.5 执行事件循环
- 初始化的最后，服务器会打印日志：accept connections
- 开始执行服务器的时间循环aeMain
4675
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
