国庆节除了陪伴吕友,花了差不多两天时间初步了解了一下Redis,接下来一段时间,将深入阅读Redis源码,也做一个记录,看一个源码,我觉得还是先从基础的模块看起,比如,数据结构,Redis中的实现了很多的数据结构,当然,很多开源项目也是自己实现各种数据结构以满足定制需求,我首先阅读的是底层字符串模块-SDS
sds的定义:
typedef char *sds;
说白了就是字符串的别名,整个Redis的字符串用的就是这个,但是内部使用了一个结构来维护这个sds
struct sdshdr {
unsigned int len;
unsigned int free;
char buf[];
};
之所以这样设计,是为了符合Redis的某一些特性,总的来说,sds相比于C语言的字符串具有以下几个优点:
1.获取字符串长度的复杂度为O(1),因为内部维护了一个保存字符串长度的len
static inline size_t sdslen(const sds s) {
struct sdshdr *sh = (void*)(s-(sizeof(struct sdshdr)));
return sh->len;
}
static inline size_t sdsavail(const sds s) {
struct sdshdr *sh = (void*)(s-(sizeof(struct sdshdr)));
return sh->free;
}
要想访问到len和free字段,还要把指针直到结构体开始出,当时看到这里还犹豫了一下,基础知识不行— —!
2.是二进制安全的
这里要解释一下什么是二进制安全,C语言中的字符串采用ASCII编码表示,但是这里我们用的是一个字符数组,怎么存进去的,拿出来还是什么,例如:test\0test\0,那么len就会是9,读出来也是test\0test\0,而不是说读到第一个结束符就结束了
sds s = sdsnewlen("test", 5);
printf("%s\n", s);
size_t len = sdslen(s);
printf("%zu\n", len);
size_t free = sdsavail(s);
printf("%zu\n", free);
可以得出得到len=5
源码:
/* Create a new sds string with the content specified by the 'init' pointer
* and 'initlen'.
* If NULL is used for 'init' the string is initialized with zero bytes.
*
* The string is always null-termined (all the sds strings are, always) so
* even if you create an sds string with:
*
* mystring = sdsnewlen("abc",3");
*
* You can print the string with printf() as there is an implicit \0 at the
* end of the string. However the string is binary safe and can contain
* \0 characters in the middle, as the length is stored in the sds header. */
sds sdsnewlen(const void *init, size_t initlen) {
struct sdshdr *sh;
if (init) {
sh = malloc(sizeof(struct sdshdr)+initlen+1);
} else {
sh = calloc(1,sizeof(struct sdshdr)+initlen+1);
}
if (sh == NULL) return NULL;
sh->len = initlen;
sh->free = 0;
if (initlen && init)
memcpy(sh->buf, init, initlen);
sh->buf[initlen] = '\0';
return (char*)sh->buf;
}
3.不会出现缓冲区溢出,利用free字段可以减少修改字符串长度导致的内存重新分配
我们知道,C语言中的这两组函数如果使用不当,是会造成缓冲区溢出的
char *strcat(char *dest, const char *src);
char *strncat(char *dest, const char *src, size_t n);
The strcat() function appends the src string to the dest string, overwriting the null byte ('\0') at the end of dest, and then adds a terminating null byte. The strings may not overlap, and the dest string must have enough space for the result.
The strncat() function is similar, except that
* it will use at most n characters from src; and
* src does not need to be null terminated if it contains n or more characters.
As with strcat(), the resulting string in dest is always null terminated.
而sds中的cat实现如下:
/* Append the specified binary-safe string pointed by 't' of 'len' bytes to the
* end of the specified sds string 's'.
*
* After the call, the passed sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdscatlen(sds s, const void *t, size_t len) {
struct sdshdr *sh;
size_t curlen = sdslen(s);
s = sdsMakeRoomFor(s,len);
if (s == NULL) return NULL;
sh = (void*) (s-(sizeof(struct sdshdr)));
memcpy(s+curlen, t, len);
sh->len = curlen+len;
sh->free = sh->free-len;
s[curlen+len] = '\0';
return s;
}
那么sdsMakeRoomFor是如何实现的呢?
/* Enlarge the free space at the end of the sds string so that the caller
* is sure that after calling this function can overwrite up to addlen
* bytes after the end of the string, plus one more byte for nul term.
*
* Note: this does not change the *length* of the sds string as returned
* by sdslen(), but only the free buffer space we have. */
sds sdsMakeRoomFor(sds s, size_t addlen) {
struct sdshdr *sh, *newsh;
size_t free = sdsavail(s);
size_t len, newlen;
if (free >= addlen) return s;
len = sdslen(s);
sh = (void*) (s-(sizeof(struct sdshdr)));
newlen = (len+addlen);
if (newlen < SDS_MAX_PREALLOC)
newlen *= 2;
else
newlen += SDS_MAX_PREALLOC;
newsh = realloc(sh, sizeof(struct sdshdr)+newlen+1);
if (newsh == NULL) return NULL;
newsh->free = newlen - len;
return newsh->buf;
}
可以看出sds对于字符串长度修改有这样一个规则,如果
AddLen < free 使用free
AddLen > free && AddLen < 1M free = len
AddLen > free && AddLen > 1M free = 1M