系列文章
对象的类型与编码
redis使用两个对象来表示一个键值对,一个对象代表键key,一个对象代表值value
typedef struct redisObject {
unsigned type:4; //类型
unsigned encoding:4; //编码
unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
* LFU data (least significant 8 bits frequency
* and most significant 16 bits access time). */
int refcount; //引用技术
void *ptr;
} robj;
这里使用引用计数来回收内存,ptr指向实际的数据结构。LRU_BITS大概是与最近的访问该对象时间相关的(此处不确定)。
type里记录的都是值对象的类型,例如string,hash,list,set,zset。可以通过命令TYPE key 来查询。
encoding记录底层数据结构类型。可以通过命令OBJECT ENCODING key 来查询。
编码方式有如下
/* Objects encoding. Some kind of objects like Strings and Hashes can be
* internally represented in multiple ways. The 'encoding' field of the object
* is set to one of this fields for this object. */
#define OBJ_ENCODING_RAW 0 /* Raw representation */
#define OBJ_ENCODING_INT 1 /* Encoded as integer */
#define OBJ_ENCODING_HT 2 /* Encoded as hash table */
#define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */
#define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */
#define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */
#define OBJ_ENCODING_INTSET 6 /* Encoded as intset */
#define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */
#define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */
#define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */
#define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */
一、字符串对象
保存的为整数,且整数可以用long表示的时候,用“int”型。
字符串在44字节以下使用 embstr,44字节以上使用sds。embstr的字符串内容就存储在object的后面。
/* Create a string object with EMBSTR encoding if it is smaller than
* OBJ_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
* used.
*
* The current limit of 44 is chosen so that the biggest string object
* we allocate as EMBSTR will still fit into the 64 byte arena of jemalloc. */
#define OBJ_ENCODING_EMBSTR_SIZE_LIMIT 44
robj *createStringObject(const char *ptr, size_t len) {
if (len <= OBJ_ENCODING_EMBSTR_SIZE_LIMIT)
return createEmbeddedStringObject(ptr,len);
else
return createRawStringObject(ptr,len);
}
127.0.0.1:6379> SET msg aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OK
127.0.0.1:6379> OBJECT ENCODING msg
“embstr” //44个a的时候,是embstr
127.0.0.1:6379> SET msg aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OK
127.0.0.1:6379> OBJECT ENCODING msg
“raw” //45个a的时候,是raw
二、列表对象
在3.2以前的版本使用的是ziplist和双向链表。新版里使用的是quicklist
127.0.0.1:6379> RPUSH msg wo
(integer) 4
127.0.0.1:6379> OBJECT ENCODING msg
“quicklist”
127.0.0.1:6379>
typedef struct quicklist {
quicklistNode *head;
quicklistNode *tail;
unsigned long count; /* total count of all entries in all ziplists */
unsigned long len; /* number of quicklistNodes */
int fill : QL_FILL_BITS; /* fill factor for individual nodes */
unsigned int compress : QL_COMP_BITS; /* depth of end nodes not to compress;0=off */
unsigned int bookmark_count: QL_BM_BITS;
quicklistBookmark bookmarks[];
} quicklist;
//数据存储在quicklistNode里面,fill用来控制每个node的大小sz
typedef struct quicklistNode {
struct quicklistNode *prev;
struct quicklistNode *next;
unsigned char *zl; //用来存储数据
unsigned int sz; /* ziplist size in bytes */
unsigned int count : 16; /* count of items in ziplist */
unsigned int encoding : 2; /* RAW==1 or LZF==2 */
unsigned int container : 2; /* NONE==1 or ZIPLIST==2 */
unsigned int recompress : 1; /* was this node previous compressed? */
unsigned int attempted_compress : 1; /* node can't compress; too small */
unsigned int extra : 10; /* more bits to steal for future usage */
} quicklistNode;
//这里需要注意的是zl里存储的数据有两种,一种是ziplist,还有一种是用LZF压缩的quicklistLZF
typedef struct quicklistLZF {
unsigned int sz; /* LZF size in bytes*/
char compressed[];
} quicklistLZF;
关于LZF这方面目前还不太了解
其他相关信息可以参考https://www.cnblogs.com/jstarseven/p/12765251.html
三. 哈希对象
哈希对象是由ziplist和hashtable实现。基本上在存储的[key,value]长度比较小的时候,以及存储的键值对数量比较少的时候,是使用ziplist。
ziplist存储方式为先存储key,再存储value。ziplist中以相连的方式存储起来
四. 集合对象
集合对象底层由intset和hashtable实现。其中intset是有序的存储整数
127.0.0.1:6379> SADD msg 2 1 3 4
(integer) 4
127.0.0.1:6379> SMEMBERS msg
1) "1"
2) "2"
3) "3"
4) "4"
当全部为整数时,使用intset。如果有字符,则换成hashtable
127.0.0.1:6379> SADD msg 1 2 3 a
(integer) 4
127.0.0.1:6379> SMEMBERS msg
1) "a"
2) "3"
3) "2"
4) "1"
127.0.0.1:6379> OBJECT ENCODING msg
"hashtable"
当用hashtable存储的时候,key设置为SADD后添加的值,dict中相应的value设置为NULL
五. 有序集合对象
有序集合使用ziplist和skiplist和dict
127.0.0.1:6379> ZSCAN msg 0
1) "0"
2) 1) "a"
2) "1"
3) "b"
4) "2"
127.0.0.1:6379> :set hints
127.0.0.1:6379> OBJECT ENCODING msg
"ziplist"
使用ziplist的时候存储方式为 key score key score这样的顺序
使用skiplist的时候,会和dict一起使用
typedef struct zset {
dict *dict;
zskiplist *zsl;
} zset;
之所以要dict和zskiplist一起使用,是因为dict可以O(1)的访问,但是如果要访问某个范围,使用ZRANGE。就需要O(NlogN)的时间。而zskiplist就只需要O(n)的时间。为了节省空间,dict和zskiplist使用的SDS是同一个
* Note that the SDS string representing the element is the same in both
* the hash table and skiplist in order to save memory. What we do in order
* to manage the shared SDS string more easily is to free the SDS string
* only in zslFreeNode(). The dictionary has no value free method set.
* So we should always remove an element from the dictionary, and later from
* the skiplist.