Redis的Object

系列文章

redis的数据结构

---

对象的类型与编码

redis使用两个对象来表示一个键值对，一个对象代表键key，一个对象代表值value

typedef struct redisObject {
    unsigned type:4;  //类型
    unsigned encoding:4;  //编码
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
                            * LFU data (least significant 8 bits frequency
                            * and most significant 16 bits access time). */
    int refcount;  //引用技术
    void *ptr;
} robj;

这里使用引用计数来回收内存，ptr指向实际的数据结构。LRU_BITS大概是与最近的访问该对象时间相关的（此处不确定）。
type里记录的都是值对象的类型，例如string，hash，list，set，zset。可以通过命令TYPE key 来查询。
encoding记录底层数据结构类型。可以通过命令OBJECT ENCODING key 来查询。
编码方式有如下

/* Objects encoding. Some kind of objects like Strings and Hashes can be
 * internally represented in multiple ways. The 'encoding' field of the object
 * is set to one of this fields for this object. */
#define OBJ_ENCODING_RAW 0     /* Raw representation */
#define OBJ_ENCODING_INT 1     /* Encoded as integer */
#define OBJ_ENCODING_HT 2      /* Encoded as hash table */
#define OBJ_ENCODING_ZIPMAP 3  /* Encoded as zipmap */
#define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */
#define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */
#define OBJ_ENCODING_INTSET 6  /* Encoded as intset */
#define OBJ_ENCODING_SKIPLIST 7  /* Encoded as skiplist */
#define OBJ_ENCODING_EMBSTR 8  /* Embedded sds string encoding */
#define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */
#define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */

---

一、字符串对象

保存的为整数，且整数可以用long表示的时候，用“int”型。
字符串在44字节以下使用 embstr，44字节以上使用sds。embstr的字符串内容就存储在object的后面。

/* Create a string object with EMBSTR encoding if it is smaller than
 * OBJ_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
 * used.
 *
 * The current limit of 44 is chosen so that the biggest string object
 * we allocate as EMBSTR will still fit into the 64 byte arena of jemalloc. */
#define OBJ_ENCODING_EMBSTR_SIZE_LIMIT 44
robj *createStringObject(const char *ptr, size_t len) {
    if (len <= OBJ_ENCODING_EMBSTR_SIZE_LIMIT)
        return createEmbeddedStringObject(ptr,len);
    else
        return createRawStringObject(ptr,len);
}

127.0.0.1:6379> SET msg aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OK
127.0.0.1:6379> OBJECT ENCODING msg
“embstr” //44个a的时候，是embstr
127.0.0.1:6379> SET msg aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OK
127.0.0.1:6379> OBJECT ENCODING msg
“raw” //45个a的时候，是raw

二、列表对象

在3.2以前的版本使用的是ziplist和双向链表。新版里使用的是quicklist

127.0.0.1:6379> RPUSH msg wo
(integer) 4
127.0.0.1:6379> OBJECT ENCODING msg
“quicklist”
127.0.0.1:6379>

typedef struct quicklist {
    quicklistNode *head;
    quicklistNode *tail;
    unsigned long count;        /* total count of all entries in all ziplists */
    unsigned long len;          /* number of quicklistNodes */
    int fill : QL_FILL_BITS;              /* fill factor for individual nodes */
    unsigned int compress : QL_COMP_BITS; /* depth of end nodes not to compress;0=off */
    unsigned int bookmark_count: QL_BM_BITS;
    quicklistBookmark bookmarks[];
} quicklist;
//数据存储在quicklistNode里面，fill用来控制每个node的大小sz
typedef struct quicklistNode {
    struct quicklistNode *prev;
    struct quicklistNode *next;
    unsigned char *zl;  //用来存储数据
    unsigned int sz;             /* ziplist size in bytes */
    unsigned int count : 16;     /* count of items in ziplist */
    unsigned int encoding : 2;   /* RAW==1 or LZF==2 */
    unsigned int container : 2;  /* NONE==1 or ZIPLIST==2 */
    unsigned int recompress : 1; /* was this node previous compressed? */
    unsigned int attempted_compress : 1; /* node can't compress; too small */
    unsigned int extra : 10; /* more bits to steal for future usage */
} quicklistNode;
//这里需要注意的是zl里存储的数据有两种，一种是ziplist，还有一种是用LZF压缩的quicklistLZF
typedef struct quicklistLZF {
    unsigned int sz; /* LZF size in bytes*/
    char compressed[];
} quicklistLZF;

关于LZF这方面目前还不太了解
其他相关信息可以参考https://www.cnblogs.com/jstarseven/p/12765251.html

---

三. 哈希对象

哈希对象是由ziplist和hashtable实现。基本上在存储的[key,value]长度比较小的时候，以及存储的键值对数量比较少的时候，是使用ziplist。
ziplist存储方式为先存储key，再存储value。ziplist中以相连的方式存储起来

四. 集合对象

集合对象底层由intset和hashtable实现。其中intset是有序的存储整数

127.0.0.1:6379> SADD msg  2 1 3 4
(integer) 4
127.0.0.1:6379> SMEMBERS msg
1) "1"
2) "2"
3) "3"
4) "4"

当全部为整数时，使用intset。如果有字符，则换成hashtable

127.0.0.1:6379> SADD msg 1 2 3 a
(integer) 4
127.0.0.1:6379> SMEMBERS msg
1) "a"
2) "3"
3) "2"
4) "1"
127.0.0.1:6379> OBJECT ENCODING msg
"hashtable"

当用hashtable存储的时候，key设置为SADD后添加的值，dict中相应的value设置为NULL

五. 有序集合对象

有序集合使用ziplist和skiplist和dict

127.0.0.1:6379> ZSCAN msg 0
1) "0"
2) 1) "a"
   2) "1"
   3) "b"
   4) "2"
127.0.0.1:6379> :set hints
127.0.0.1:6379> OBJECT ENCODING msg
"ziplist"

使用ziplist的时候存储方式为 key score key score这样的顺序
使用skiplist的时候，会和dict一起使用

typedef struct zset {
    dict *dict;
    zskiplist *zsl;
} zset;

之所以要dict和zskiplist一起使用，是因为dict可以O(1)的访问，但是如果要访问某个范围，使用ZRANGE。就需要O(NlogN）的时间。而zskiplist就只需要O（n）的时间。为了节省空间，dict和zskiplist使用的SDS是同一个

 * Note that the SDS string representing the element is the same in both
 * the hash table and skiplist in order to save memory. What we do in order
 * to manage the shared SDS string more easily is to free the SDS string
 * only in zslFreeNode(). The dictionary has no value free method set.
 * So we should always remove an element from the dictionary, and later from
 * the skiplist.