一:Redis是如何处理请求的?
Redis使用单线程模型处理并发请求。具体来说,Redis通过事件驱动模型来实现并发处理。
在Redis的源码中,主要涉及到以下几个关键的组件和概念:
事件处理器(EventLoop):Redis使用一个单独的事件处理器来监听和处理各种事件,例如客户端请求、定时器事件等。事件处理器是Redis的核心组件之一。
文件事件(File Event):Redis使用文件事件来处理网络请求。文件事件是一种基于操作系统提供的多路复用技术(如epoll、kqueue等)的抽象,可以同时监听多个套接字的可读、可写状态。
命令请求队列(Command Request Queue):Redis将客户端的请求放入一个队列中,等待事件处理器处理。请求队列使用先进先出(FIFO)的方式,保证请求的顺序。
事件分派器(Event Dispatcher):事件分派器会将队列中的请求逐个分派给事件处理器处理。事件分派器会按照一定的策略**(如轮询、随机等)选择一个事件处理器来处理请求**。
命令执行器(Command Executor):事件处理器会从请求队列中取出请求,并执行相应的命令。命令执行过程中,Redis会进行数据读写、计算等操作。
通过以上组件和概念的配合,Redis能够实现高并发的请求处理。单线程的特点使得Redis能够避免多线程的竞争和锁等开销,从而提高请求处理的效率。同时,Redis的事件驱动模型也能够有效地处理异步事件,提供高性能的IO处理能力。
二:Redis一条命令串讲
以redis命令SET KEY VALUE为例子,将整个过程串通。
2.1 将redis编译
在Redis make install编译完成后,会生成以下几个主要的文件和目录:
make install后,会编译这几个部分
redis-server:这是Redis服务器的可执行文件。它负责启动Redis服务器端并监听指定的端口,等待来自客户端的连接和命令。
redis-cli:这是Redis的命令行客户端工具。它可以用于与Redis服务器进行交互,执行各种Redis命令,发送和接收数据。
redis-benchmark:这是Redis的基准测试工具。它可以用于测试Redis服务器的性能和吞吐量,模拟并发请求和负载。
redis-check-aof:这是Redis的AOF日志文件检查工具。它用于检查和修复AOF日志文件以确保数据的完整性和一致性。
redis-check-rdb:这是Redis的RDB快照文件检查工具。它用于检查RDB快照文件以确保数据的完整性和一致性。
redis.conf:这是Redis的默认配置文件。它包含了Redis服务器的各种配置选项,例如监听的端口、数据存储路径、日志设置等。
src目录:这是Redis源代码的主要目录。它包含了Redis的核心代码和各种模块,如网络模块、存储模块、命令解析模块等。
deps目录:这是Redis的依赖库目录。Redis依赖于一些第三方库,如Jemalloc、Lua等,这些库会被编译和链接到Redis中。
以上是Redis编译后生成的一些主要文件和目录。具体生成的文件和目录结构可能会因Redis版本和编译选项的不同而有所变化。
2.2 redis-server端准备
redis的main函数,在命令行敲redis-server,会执行src/server.c文件的main函数,启动redis的服务端。这块代码
部分我添加了注释
int main(int argc, char **argv) {
struct timeval tv;
int j;
char config_from_stdin = 0;
#ifdef REDIS_TEST
monotonicInit(); /* Required for dict tests, that are relying on monotime during dict rehashing. */
if (argc >= 3 && !strcasecmp(argv[1], "test")) {
int flags = 0;
for (j = 3; j < argc; j++) {
char *arg = argv[j];
if (!strcasecmp(arg, "--accurate")) flags |= REDIS_TEST_ACCURATE;
else if (!strcasecmp(arg, "--large-memory")) flags |= REDIS_TEST_LARGE_MEMORY;
else if (!strcasecmp(arg, "--valgrind")) flags |= REDIS_TEST_VALGRIND;
}
if (!strcasecmp(argv[2], "all")) {
int numtests = sizeof(redisTests)/sizeof(struct redisTest);
for (j = 0; j < numtests; j++) {
redisTests[j].failed = (redisTests[j].proc(argc,argv,flags) != 0);
}
/* Report tests result */
int failed_num = 0;
for (j = 0; j < numtests; j++) {
if (redisTests[j].failed) {
failed_num++;
printf("[failed] Test - %s\n", redisTests[j].name);
} else {
printf("[ok] Test - %s\n", redisTests[j].name);
}
}
printf("%d tests, %d passed, %d failed\n", numtests,
numtests-failed_num, failed_num);
return failed_num == 0 ? 0 : 1;
} else {
redisTestProc *proc = getTestProcByName(argv[2]);
if (!proc) return -1; /* test not found */
return proc(argc,argv,flags);
}
return 0;
}
#endif
/* We need to initialize our libraries, and the server configuration. */
#ifdef INIT_SETPROCTITLE_REPLACEMENT
spt_init(argc, argv);
#endif
tzset(); /* Populates 'timezone' global. */
zmalloc_set_oom_handler(redisOutOfMemoryHandler);
/* To achieve entropy, in case of containers, their time() and getpid() can
* be the same. But value of tv_usec is fast enough to make the difference */
gettimeofday(&tv,NULL); //获取当前时间戳
srand(time(NULL)^getpid()^tv.tv_usec); //初始化随机种子
srandom(time(NULL)^getpid()^tv.tv_usec); //初始化随机种子
init_genrand64(((long long) tv.tv_sec * 1000000 + tv.tv_usec) ^ getpid()); //初始化Redis自己实现的随机数生成器
crc64_init(); //初始化CRC64校验
/* Store umask value. Because umask(2) only offers a set-and-get API we have
* to reset it and restore it back. We do this early to avoid a potential
* race condition with threads that could be creating files or directories.
*/
umask(server.umask = umask(0777)); //设置新创建的文件和目录的默认权限。
uint8_t hashseed[16];
getRandomBytes(hashseed,sizeof(hashseed));
dictSetHashFunctionSeed(hashseed);
char *exec_name = strrchr(argv[0], '/');
if (exec_name == NULL) exec_name = argv[0];
server.sentinel_mode = checkForSentinelMode(argc,argv, exec_name);
initServerConfig();
ACLInit(); /* The ACL subsystem must be initialized ASAP because the
basic networking code and client creation depends on it. */
moduleInitModulesSystem();
connTypeInitialize();
/* Store the executable path and arguments in a safe place in order
* to be able to restart the server later. */
server.executable = getAbsolutePath(argv[0]);
server.exec_argv = zmalloc(sizeof(char*)*(argc+1));
server.exec_argv[argc] = NULL;
for (j = 0; j < argc; j++) server.exec_argv[j] = zstrdup(argv[j]); //给命令行参数赋空间,把命令行的内容复制到server.exec_argv上
/* We need to init sentinel right now as parsing the configuration file
* in sentinel mode will have the effect of populating the sentinel
* data structures with master nodes to monitor. */
if (server.sentinel_mode) {
initSentinelConfig();
initSentinel();
}
/* Check if we need to start in redis-check-rdb/aof mode. We just execute
* the program main. However the program is part of the Redis executable
* so that we can easily execute an RDB check on loading errors. */
if (strstr(exec_name,"redis-check-rdb") != NULL)
redis_check_rdb_main(argc,argv,NULL);
else if (strstr(exec_name,"redis-check-aof") != NULL)
redis_check_aof_main(argc,argv);
if (argc >= 2) {
j = 1; /* First option to parse in argv[] */
sds options = sdsempty();
/* Handle special options --help and --version */
if (strcmp(argv[1], "-v") == 0 ||
strcmp(argv[1], "--version") == 0) version();
if (strcmp(argv[1], "--help") == 0 ||
strcmp(argv[1], "-h") == 0) usage();
if (strcmp(argv[1], "--test-memory") == 0) {
if (argc == 3) {
memtest(atoi(argv[2]),50);
exit(0);
} else {
fprintf(stderr,"Please specify the amount of memory to test in megabytes.\n");
fprintf(stderr,"Example: ./redis-server --test-memory 4096\n\n");
exit(1);
}
} if (strcmp(argv[1], "--check-system") == 0) {
exit(syscheck() ? 0 : 1);
}
/* Parse command line options
* Precedence wise, File, stdin, explicit options -- last config is the one that matters.
*
* First argument is the config file name? */
if (argv[1][0] != '-') {
/* Replace the config file in server.exec_argv with its absolute path. */
server.configfile = getAbsolutePath(argv[1]);
zfree(server.exec_argv[1]);
server.exec_argv[1] = zstrdup(server.configfile);
j = 2; // Skip this arg when parsing options
}
sds *argv_tmp;
int argc_tmp;
int handled_last_config_arg = 1;
while(j < argc) {
/* Either first or last argument - Should we read config from stdin? */
if (argv[j][0] == '-' && argv[j][1] == '\0' && (j == 1 || j == argc-1)) {
config_from_stdin = 1;
}
/* All the other options are parsed and conceptually appended to the
* configuration file. For instance --port 6380 will generate the
* string "port 6380\n" to be parsed after the actual config file
* and stdin input are parsed (if they exist).
* Only consider that if the last config has at least one argument. */
else if (handled_last_config_arg && argv[j][0] == '-' && argv[j][1] == '-') {
/* Option name */
if (sdslen(options)) options = sdscat(options,"\n");
/* argv[j]+2 for removing the preceding `--` */
options = sdscat(options,argv[j]+2);
options = sdscat(options," ");
argv_tmp = sdssplitargs(argv[j], &argc_tmp);
if (argc_tmp == 1) {
/* Means that we only have one option name, like --port or "--port " */
handled_last_config_arg = 0;
if ((j != argc-1) && argv[j+1][0] == '-' && argv[j+1][1] == '-' &&
!strcasecmp(argv[j], "--save"))
{
/* Special case: handle some things like `--save --config value`.
* In this case, if next argument starts with `--`, we will reset
* handled_last_config_arg flag and append an empty "" config value
* to the options, so it will become `--save "" --config value`.
* We are doing it to be compatible with pre 7.0 behavior (which we
* break it in #10660, 7.0.1), since there might be users who generate
* a command line from an array and when it's empty that's what they produce. */
options = sdscat(options, "\"\"");
handled_last_config_arg = 1;
}
else if ((j == argc-1) && !strcasecmp(argv[j], "--save")) {
/* Special case: when empty save is the last argument.
* In this case, we append an empty "" config value to the options,
* so it will become `--save ""` and will follow the same reset thing. */
options = sdscat(options, "\"\"");
}
else if ((j != argc-1) && argv[j+1][0] == '-' && argv[j+1][1] == '-' &&
!strcasecmp(argv[j], "--sentinel"))
{
/* Special case: handle some things like `--sentinel --config value`.
* It is a pseudo config option with no value. In this case, if next
* argument starts with `--`, we will reset handled_last_config_arg flag.
* We are doing it to be compatible with pre 7.0 behavior (which we
* break it in #10660, 7.0.1). */
options = sdscat(options, "");
handled_last_config_arg = 1;
}
else if ((j == argc-1) && !strcasecmp(argv[j], "--sentinel")) {
/* Special case: when --sentinel is the last argument.
* It is a pseudo config option with no value. In this case, do nothing.
* We are doing it to be compatible with pre 7.0 behavior (which we
* break it in #10660, 7.0.1). */
options = sdscat(options, "");
}
} else {
/* Means that we are passing both config name and it's value in the same arg,
* like "--port 6380", so we need to reset handled_last_config_arg flag. */
handled_last_config_arg = 1;
}
sdsfreesplitres(argv_tmp, argc_tmp);
} else {
/* Option argument */
options = sdscatrepr(options,argv[j],strlen(argv[j]));
options = sdscat(options," ");
handled_last_config_arg = 1;
}
j++;
}
loadServerConfig(server.configfile, config_from_stdin, options);
if (server.sentinel_mode) loadSentinelConfigFromQueue();
sdsfree(options);
}
if (server.sentinel_mode) sentinelCheckConfigFile();
/* Do system checks */
#ifdef __linux__
linuxMemoryWarnings();
sds err_msg = NULL;
if (checkXenClocksource(&err_msg) < 0) {
serverLog(LL_WARNING, "WARNING %s", err_msg);
sdsfree(err_msg);
}
#if defined (__arm64__)
int ret;
if ((ret = checkLinuxMadvFreeForkBug(&err_msg)) <= 0) {
if (ret < 0) {
serverLog(LL_WARNING, "WARNING %s", err_msg);
sdsfree(err_msg);
} else
serverLog(LL_WARNING, "Failed to test the kernel for a bug that could lead to data corruption during background save. "
"Your system could be affected, please report this error.");
if (!checkIgnoreWarning("ARM64-COW-BUG")) {
serverLog(LL_WARNING,"Redis will now exit to prevent data corruption. "
"Note that it is possible to suppress this warning by setting the following config: ignore-warnings ARM64-COW-BUG");
exit(1);
}
}
#endif /* __arm64__ */
#endif /* __linux__ */
/* Daemonize if needed */
server.supervised = redisIsSupervised(server.supervised_mode);
int background = server.daemonize && !server.supervised;
if (background) daemonize();
serverLog(LL_NOTICE, "oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo");
serverLog(LL_NOTICE,
"Redis version=%s, bits=%d, commit=%s, modified=%d, pid=%d, just started",
REDIS_VERSION,
(sizeof(long) == 8) ? 64 : 32,
redisGitSHA1(),
strtol(redisGitDirty(),NULL,10) > 0,
(int)getpid());
if (argc == 1) {
serverLog(LL_WARNING, "Warning: no config file specified, using the default config. In order to specify a config file use %s /path/to/redis.conf", argv[0]);
} else {
serverLog(LL_NOTICE, "Configuration loaded");
}
initServer();
if (background || server.pidfile) createPidFile();
if (server.set_proc_title) redisSetProcTitle(NULL);
redisAsciiArt();
checkTcpBacklogSettings();
if (server.cluster_enabled) {
clusterInit();
}
if (!server.sentinel_mode) {
moduleInitModulesSystemLast();
moduleLoadFromQueue();
}
ACLLoadUsersAtStartup();
initListeners();
if (server.cluster_enabled) {
clusterInitLast();
}
InitServerLast();
if (!server.sentinel_mode) {
/* Things not needed when running in Sentinel mode. */
serverLog(LL_NOTICE,"Server initialized");
aofLoadManifestFromDisk();
loadDataFromDisk();
aofOpenIfNeededOnServerStart();
aofDelHistoryFiles();
if (server.cluster_enabled) {
serverAssert(verifyClusterConfigWithData() == C_OK);
}
for (j = 0; j < CONN_TYPE_MAX; j++) {
connListener *listener = &server.listeners[j];
if (listener->ct == NULL)
continue;
serverLog(LL_NOTICE,"Ready to accept connections %s", listener->ct->get_type(NULL));
}
if (server.supervised_mode == SUPERVISED_SYSTEMD) {
if (!server.masterhost) {
redisCommunicateSystemd("STATUS=Ready to accept connections\n");
} else {
redisCommunicateSystemd("STATUS=Ready to accept connections in read-only mode. Waiting for MASTER <-> REPLICA sync\n");
}
redisCommunicateSystemd("READY=1\n");
}
} else {
sentinelIsRunning();
if (server.supervised_mode == SUPERVISED_SYSTEMD) {
redisCommunicateSystemd("STATUS=Ready to accept connections\n");
redisCommunicateSystemd("READY=1\n");
}
}
/* Warning the user about suspicious maxmemory setting. */
if (server.maxmemory > 0 && server.maxmemory < 1024*1024) {
serverLog(LL_WARNING,"WARNING: You specified a maxmemory value that is less than 1MB (current value is %llu bytes). Are you sure this is what you really want?", server.maxmemory);
}
redisSetCpuAffinity(server.server_cpulist);
setOOMScoreAdj(-1);
aeMain(server.el);
aeDeleteEventLoop(server.el);
return 0;
}
server端整体启动流程说明:
1.使用默认配置值初始化服务器,然后从redis.conf加载配置。
2.如果在后台模式下运行,则进行Daemonize。
3.记录Redis版本和其他信息。
4.初始化模块、集群、ACL、监听器等。
5.如果不处于sentinel模式,请从磁盘加载数据,如果需要,请启动AOF,验证集群配置。
6.记录服务器已准备好接受连接。
7.使用aeMain()进入主事件循环。这是Redis等待连接/命令的地方。
8.当aeMain()退出时,清除事件循环。
重点在最后一句,aeMain进入主事件循环。现在,先将视角切到redis-cli客户端上
2.3 redis-cli端与redis-server进行连接
int main(int argc, char **argv) {
int firstarg;
struct timeval tv;
memset(&config.sslconfig, 0, sizeof(config.sslconfig));
config.conn_info.hostip = sdsnew("127.0.0.1");
config.conn_info.hostport = 6379;
config.hostsocket = NULL;
config.repeat = 1;
config.interval = 0;
config.dbnum = 0;
config.conn_info.input_dbnum = 0;
config.interactive = 0;
config.shutdown = 0;
config.monitor_mode = 0;
config.pubsub_mode = 0;
config.blocking_state_aborted = 0;
config.latency_mode = 0;
config.latency_dist_mode = 0;
config.latency_history = 0;
config.lru_test_mode = 0;
config.lru_test_sample_size = 0;
config.cluster_mode = 0;
config.cluster_send_asking = 0;
config.slave_mode = 0;
config.getrdb_mode = 0;
config.get_functions_rdb_mode = 0;
config.stat_mode = 0;
config.scan_mode = 0;
config.count = 10;
config.intrinsic_latency_mode = 0;
config.pattern = NULL;
config.rdb_filename = NULL;
config.pipe_mode = 0;
config.pipe_timeout = REDIS_CLI_DEFAULT_PIPE_TIMEOUT;
config.bigkeys = 0;
config.memkeys = 0;
config.hotkeys = 0;
config.stdin_lastarg = 0;
config.stdin_tag_arg = 0;
config.stdin_tag_name = NULL;
config.conn_info.auth = NULL;
config.askpass = 0;
config.conn_info.user = NULL;
config.eval = NULL;
config.eval_ldb = 0;
config.eval_ldb_end = 0;
config.eval_ldb_sync = 0;
config.enable_ldb_on_eval = 0;
config.last_cmd_type = -1;
config.last_reply = NULL;
config.verbose = 0;
config.set_errcode = 0;
config.no_auth_warning = 0;
config.in_multi = 0;
config.server_version = NULL;
config.cluster_manager_command.name = NULL;
config.cluster_manager_command.argc = 0;
config.cluster_manager_command.argv = NULL;
config.cluster_manager_command.stdin_arg = NULL;
config.cluster_manager_command.flags = 0;
config.cluster_manager_command.replicas = 0;
config.cluster_manager_command.from = NULL;
config.cluster_manager_command.to = NULL;
config.cluster_manager_command.from_user = NULL;
config.cluster_manager_command.from_pass = NULL;
config.cluster_manager_command.from_askpass = 0;
config.cluster_manager_command.weight = NULL;
config.cluster_manager_command.weight_argc = 0;
config.cluster_manager_command.slots = 0;
config.cluster_manager_command.timeout = CLUSTER_MANAGER_MIGRATE_TIMEOUT;
config.cluster_manager_command.pipeline = CLUSTER_MANAGER_MIGRATE_PIPELINE;
config.cluster_manager_command.threshold =
CLUSTER_MANAGER_REBALANCE_THRESHOLD;
config.cluster_manager_command.backup_dir = NULL;
pref.hints = 1;
spectrum_palette = spectrum_palette_color;
spectrum_palette_size = spectrum_palette_color_size;
if (!isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL)) {
config.output = OUTPUT_RAW;
config.push_output = 0;
} else {
config.output = OUTPUT_STANDARD;
config.push_output = 1;
}
config.mb_delim = sdsnew("\n");
config.cmd_delim = sdsnew("\n");
firstarg = parseOptions(argc,argv);
argc -= firstarg;
argv += firstarg;
parseEnv();
if (config.askpass) {
config.conn_info.auth = askPassword("Please input password: ");
}
if (config.cluster_manager_command.from_askpass) {
config.cluster_manager_command.from_pass = askPassword(
"Please input import source node password: ");
}
#ifdef USE_OPENSSL
if (config.tls) {
cliSecureInit();
}
#endif
gettimeofday(&tv, NULL);
init_genrand64(((long long) tv.tv_sec * 1000000 + tv.tv_usec) ^ getpid());
/* Cluster Manager mode */
if (CLUSTER_MANAGER_MODE()) {
clusterManagerCommandProc *proc = validateClusterManagerCommand();
if (!proc) {
exit(1);
}
clusterManagerMode(proc);
}
/* Latency mode */
if (config.latency_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
latencyMode();
}
/* Latency distribution mode */
if (config.latency_dist_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
latencyDistMode();
}
/* Slave mode */
if (config.slave_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
sendCapa();
sendReplconf("rdb-filter-only", "");
slaveMode(1);
}
/* Get RDB/functions mode. */
if (config.getrdb_mode || config.get_functions_rdb_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
sendCapa();
sendRdbOnly();
if (config.get_functions_rdb_mode && !sendReplconf("rdb-filter-only", "functions")) {
fprintf(stderr, "Failed requesting functions only RDB from server, aborting\n");
exit(1);
}
getRDB(NULL);
}
/* Pipe mode */
if (config.pipe_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
pipeMode();
}
/* Find big keys */
if (config.bigkeys) {
if (cliConnect(0) == REDIS_ERR) exit(1);
findBigKeys(0, 0);
}
/* Find large keys */
if (config.memkeys) {
if (cliConnect(0) == REDIS_ERR) exit(1);
findBigKeys(1, config.memkeys_samples);
}
/* Find hot keys */
if (config.hotkeys) {
if (cliConnect(0) == REDIS_ERR) exit(1);
findHotKeys();
}
/* Stat mode */
if (config.stat_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
if (config.interval == 0) config.interval = 1000000;
statMode();
}
/* Scan mode */
if (config.scan_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
scanMode();
}
/* LRU test mode */
if (config.lru_test_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
LRUTestMode();
}
/* Intrinsic latency mode */
if (config.intrinsic_latency_mode) intrinsicLatencyMode();
/* Print command-line hint for an input prefix string */
if (config.test_hint) {
testHint(config.test_hint);
}
/* Run test suite for command-line hints */
if (config.test_hint_file) {
testHintSuite(config.test_hint_file);
}
/* Start interactive mode when no command is provided */
if (argc == 0 && !config.eval) {
/* Ignore SIGPIPE in interactive mode to force a reconnect */
signal(SIGPIPE, SIG_IGN);
signal(SIGINT, sigIntHandler);
/* Note that in repl mode we don't abort on connection error.
* A new attempt will be performed for every command send. */
cliConnect(0);
repl();
}
/* Otherwise, we have some arguments to execute */
if (config.eval) {
if (cliConnect(0) != REDIS_OK) exit(1);
return evalMode(argc,argv);
} else {
cliConnect(CC_QUIET);
return noninteractive(argc,argv);
}
}
重点是这几个函数:
- connectWithTimeout(): 用于连接Redis服务器,建立TCP连接。
- redisAsyncCommand(): 用于发送命令给Redis服务器。
- redisBufferReadReply(): 用于从服务器接收响应。
- redisAppendFormattedCommand(): 用于构建符合Redis协议的命令字符串。
redis用hiredis.c函数库,用于与redis-server端进行通信。
在redis-cli最后代码,有cliConnect,这里面的代码是redis与server端通信。
1、redis-cli.cliConnect函数内调用=>hiredis.redisConnectUnix=>net.c._redisContextConnectTcp代码实现与host端的套接字通信,cliConnect函数主要是用来建立与Redis服务器的连接。
2、首先通过redisConnectUnix函数连接Unix套接字,如果没有错误并且配置了TLS,则通过cliSecureConnection函数建立TLS连接。
3、然后检查连接是否发生错误,如果发生错误,则输出错误信息并释放连接。
4、接下来设置Redis连接的KEEP_ALIVE选项,以防止长时间执行命令导致超时。
5、然后进行认证、选择数据库和切换协议的操作。最后,如果配置了push_output选项,则设置推送处理程序。最后返回连接状态。
net.c代码:
static int _redisContextConnectTcp(redisContext *c, const char *addr, int port,
const struct timeval *timeout,
const char *source_addr) {
redisFD s;
int rv, n;
char _port[6]; /* strlen("65535"); */
struct addrinfo hints, *servinfo, *bservinfo, *p, *b;
int blocking = (c->flags & REDIS_BLOCK);
int reuseaddr = (c->flags & REDIS_REUSEADDR);
int reuses = 0;
long timeout_msec = -1;
servinfo = NULL;
c->connection_type = REDIS_CONN_TCP;
c->tcp.port = port;
/* We need to take possession of the passed parameters
* to make them reusable for a reconnect.
* We also carefully check we don't free data we already own,
* as in the case of the reconnect method.
*
* This is a bit ugly, but atleast it works and doesn't leak memory.
**/
if (c->tcp.host != addr) {
hi_free(c->tcp.host);
c->tcp.host = hi_strdup(addr);
if (c->tcp.host == NULL)
goto oom;
}
if (timeout) {
if (redisContextUpdateConnectTimeout(c, timeout) == REDIS_ERR)
goto oom;
} else {
hi_free(c->connect_timeout);
c->connect_timeout = NULL;
}
if (redisContextTimeoutMsec(c, &timeout_msec) != REDIS_OK) {
goto error;
}
if (source_addr == NULL) {
hi_free(c->tcp.source_addr);
c->tcp.source_addr = NULL;
} else if (c->tcp.source_addr != source_addr) {
hi_free(c->tcp.source_addr);
c->tcp.source_addr = hi_strdup(source_addr);
}
snprintf(_port, 6, "%d", port);
memset(&hints,0,sizeof(hints));
hints.ai_family = AF_INET;
hints.ai_socktype = SOCK_STREAM;
/* DNS lookup. To use dual stack, set both flags to prefer both IPv4 and
* IPv6. By default, for historical reasons, we try IPv4 first and then we
* try IPv6 only if no IPv4 address was found. */
if (c->flags & REDIS_PREFER_IPV6 && c->flags & REDIS_PREFER_IPV4)
hints.ai_family = AF_UNSPEC;
else if (c->flags & REDIS_PREFER_IPV6)
hints.ai_family = AF_INET6;
else
hints.ai_family = AF_INET;
rv = getaddrinfo(c->tcp.host, _port, &hints, &servinfo);
if (rv != 0 && hints.ai_family != AF_UNSPEC) {
/* Try again with the other IP version. */
hints.ai_family = (hints.ai_family == AF_INET) ? AF_INET6 : AF_INET;
rv = getaddrinfo(c->tcp.host, _port, &hints, &servinfo);
}
if (rv != 0) {
__redisSetError(c, REDIS_ERR_OTHER, gai_strerror(rv));
return REDIS_ERR;
}
for (p = servinfo; p != NULL; p = p->ai_next) {
addrretry:
if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == REDIS_INVALID_FD)
continue;
c->fd = s;
if (redisSetBlocking(c,0) != REDIS_OK)
goto error;
if (c->tcp.source_addr) {
int bound = 0;
/* Using getaddrinfo saves us from self-determining IPv4 vs IPv6 */
if ((rv = getaddrinfo(c->tcp.source_addr, NULL, &hints, &bservinfo)) != 0) {
char buf[128];
snprintf(buf,sizeof(buf),"Can't get addr: %s",gai_strerror(rv));
__redisSetError(c,REDIS_ERR_OTHER,buf);
goto error;
}
if (reuseaddr) {
n = 1;
if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (char*) &n,
sizeof(n)) < 0) {
freeaddrinfo(bservinfo);
goto error;
}
}
for (b = bservinfo; b != NULL; b = b->ai_next) {
if (bind(s,b->ai_addr,b->ai_addrlen) != -1) {
bound = 1;
break;
}
}
freeaddrinfo(bservinfo);
if (!bound) {
char buf[128];
snprintf(buf,sizeof(buf),"Can't bind socket: %s",strerror(errno));
__redisSetError(c,REDIS_ERR_OTHER,buf);
goto error;
}
}
/* For repeat connection */
hi_free(c->saddr);
c->saddr = hi_malloc(p->ai_addrlen);
if (c->saddr == NULL)
goto oom;
memcpy(c->saddr, p->ai_addr, p->ai_addrlen);
c->addrlen = p->ai_addrlen;
if (connect(s,p->ai_addr,p->ai_addrlen) == -1) {
if (errno == EHOSTUNREACH) {
redisNetClose(c);
continue;
} else if (errno == EINPROGRESS) {
if (blocking) {
goto wait_for_ready;
}
/* This is ok.
* Note that even when it's in blocking mode, we unset blocking
* for `connect()`
*/
} else if (errno == EADDRNOTAVAIL && reuseaddr) {
if (++reuses >= REDIS_CONNECT_RETRIES) {
goto error;
} else {
redisNetClose(c);
goto addrretry;
}
} else {
wait_for_ready:
if (redisContextWaitReady(c,timeout_msec) != REDIS_OK)
goto error;
if (redisSetTcpNoDelay(c) != REDIS_OK)
goto error;
}
}
if (blocking && redisSetBlocking(c,1) != REDIS_OK)
goto error;
c->flags |= REDIS_CONNECTED;
rv = REDIS_OK;
goto end;
}
if (p == NULL) {
char buf[128];
snprintf(buf,sizeof(buf),"Can't create socket: %s",strerror(errno));
__redisSetError(c,REDIS_ERR_OTHER,buf);
goto error;
}
oom:
__redisSetError(c, REDIS_ERR_OOM, "Out of memory");
error:
rv = REDIS_ERR;
end:
if(servinfo) {
freeaddrinfo(servinfo);
}
return rv; // Need to return REDIS_OK if alright
}
这段代码是一个连接到Redis服务器的函数。它的主要功能是通过给定的主机名和端口号创建一个TCP连接,并尝试连接到Redis服务器。
首先,它使用getaddrinfo函数获取与主机名和端口号匹配的地址信息。然后,它遍历获取到的地址列表,尝试连接到每个地址,直到成功连接或遍历完所有地址为止。
在每次尝试连接之前,它会先关闭之前的连接(如果有的话)。然后,它创建一个新的套接字,并绑定到本地地址(如果指定了本地地址),然后尝试连接到目标地址。如果连接失败,它会根据不同的错误类型采取不同的操作:如果是主机不可达错误(EHOSTUNREACH),则关闭连接并尝试下一个地址;如果是连接正在进行中的错误(EINPROGRESS),则等待连接就绪或超时;如果是地址不可用错误(EADDRNOTAVAIL),则根据是否启用了地址重用选项进行重试。
如果连接成功,它会设置连接的一些属性(例如,设置非阻塞模式和TCP_NODELAY选项),然后将连接标记为已连接状态,并返回REDIS_OK。
如果在连接过程中出现错误,它会设置错误信息,并返回REDIS_ERR。
最后,它会释放获取到的地址信息,并返回连接的结果。
getaddrinfo函数的代码通常在操作系统的网络库中实现,例如在Linux中是在glibc库中。
getaddrinfo函数的作用是将主机名和服务名(或端口号)解析为一个或多个与之匹配的网络地址。它可以根据给定的主机名和服务名(或端口号),以及一些附加的标志和选项,返回一个或多个addrinfo结构体,每个结构体包含一个网络地址的相关信息,例如IP地址、协议族、套接字类型等。
通过调用getaddrinfo函数,我们可以将主机名和端口号转换为一个或多个用于网络通信的地址,然后使用这些地址进行TCP或UDP连接,或者进行其他网络操作。这使得我们可以编写通用的网络代码,而不需要关注不同操作系统和网络协议的细节。
最后调用connect的系统调用命令,实现TCP连接。
int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen);
2.4 redis-cli端对redis-server进行发送命令
在redis-cli.c源码文件中,解析Redis命令并发送给服务器的主要函数是void repl() ,该函数位于文件的最下方。
我们走的是这段逻辑。
如果我们没有输入任何命令(redis-cli get key value),只有redis-cli,那么开始进入交互模式。
在cliSendCommand函数中,将argv进行输出。redisAppendCommandArgv将命令格式化,cliReadReply等待事件返回。
redisFormatSdsCommandArgv解析set key value命令,发送给服务端。
如果是set mykey myvalue,最终结果为:
*3\r\n$3\r\nSET\r\n$5\r\nmykey\r\n$7\r\nmyvalue\r\n
通过
/* Construct command */
cmd = hi_sdscatfmt(cmd, "*%i\r\n", argc);
for (j=0; j < argc; j++) {
len = argvlen ? argvlen[j] : strlen(argv[j]);
cmd = hi_sdscatfmt(cmd, "$%U\r\n", len);
cmd = hi_sdscatlen(cmd, argv[j], len);
cmd = hi_sdscatlen(cmd, "\r\n", sizeof("\r\n")-1);
}
这一段来构建sds发送到server端的代码
预分配空间公式:
/* Calculate our total size */
totlen = 1+countDigits(argc)+2; // *3\r\n
for (j = 0; j < argc; j++) { //变量参数mykey, myvalue
len = argvlen ? argvlen[j] : strlen(argv[j]);
//以set为例子,$3\r\nSET\r\n $占一个字符,\r\n有两个占两字符,len代表SET长度为3,countDigits,其实就是计算len("SET")获得的长度“3”,用字符串会占多少字节
totlen += bulklen(len);
}
初始化总大小变量totlen为1+countDigits(argc)+2。这里的1表示命令字符串的第一个字符*,countDigits(argc)表示参数个数argc的位数,2表示参数个数的后面的两个字符<CR><LF>。
在用户输入命令后调用redisCommand函数来向服务器发送命令,并通过redisGetReply函数获取服务器的响应。redisAppendCommandArgv将命令保存在缓冲区当中(可批量)
整体流程:repl => issueCommandRepeat => cliSendCommand => redisAppendCommandArgv(redis解析set key value放到缓冲区) => redisBufferWrite => cliReadReply(output_raw) => redisGetReply => redisBufferWrite(将缓存区数据发送给服务端)
write到socket上。
2.5 redis-server端解析命令,将数据组装返回给redis-cli
总过程
redis-cli将写命令set key value发送到服务端的eventloop当中,Redis会将相应的事件添加到aeEventLoop中。redis-server会在aeMain的事件循环中触发获取事件的解析。eventLoop->stop = 0就会持续循环。aeProcessEvents函数会将对应读时间、写事件绑定函数进行处理。
事件掩码:
eventLoop->events[i].mask用于表示事件的类型或状态。在Redis服务器的事件循环中,每个事件都有一个对应的掩码,用于标识该事件的类型或状态。掩码通常是一个位掩码,每个位代表一种特定的事件类型或状态。
eventLoop->events[i].mask中的掩码可以包含以下事件类型或状态之一:
AE_READABLE:表示该事件是可读事件,即该事件对应的文件描述符可以从中读取数据。
AE_WRITABLE:表示该事件是可写事件,即该事件对应的文件描述符可以写入数据。
AE_ERROR:表示该事件是错误事件,即该事件对应的文件描述符发生错误。
AE_HUP:表示该事件是挂起事件,即该事件对应的文件描述符被挂起。
AE_NOMORE:表示不再监听该事件。
通过检查eventLoop->events[i].mask的值,可以确定事件的类型或状态,并采取相应的操作,如读取数据、写入数据或处理错误。
接受客户端命令,放入事件循环中
在server.c当中,main函数启动时就调用了initListeners();代码。用于监听客户端的命令。server.listeners[conn_index];已经预制好了连接类型。用CT_Socket进行注册。在connectionType的抽象类中,调用connListen,int (*listen)(connListener *listener);代表输入为listener,输出为int。
CT_Socket绑定的函数为listenToPort,调用anetTcpServer创建一个IPV4监听套接字(调用系统调用socket),并将返回的套接字文件描述符保存在sfd->fd[sfd->count]中。
调用createSocketAcceptHandler创建事件处理器。
调用connListen持续监听端口。将connectionType的accept_handler进行绑定。
这部分代码主要具体函数是aeCreateFileEvent。
anetTcpAccept 接受tcp连接。
acceptCommonHandler将事件放入就绪队列中。
写事件处理
三:数据结构优化
sds(简单动态字符串)在性能上比c++的std::string可能更好的原因如下:
预分配内存:sds在创建字符串时就会根据字符串的长度预分配足够的内存空间,避免了频繁的内存分配和释放操作。而std::string在每次字符串长度变化时都需要重新分配内存,这会引入额外的开销。
惰性释放内存:sds在字符串缩短时,并不会立即释放多余的内存,而是将其保留以备后续使用。这样就避免了频繁的内存分配和释放操作,提高了性能。而std::string在缩短字符串时会立即释放多余的内存,这可能导致频繁的内存分配和释放操作。