基于Clion的Redis源码Stream剖析及一些思考

前言

redis是一个key-value存储系统。和Memcached类似，它支持存储的value类型相对更多，包括string、list、set、zset，stream，hyperloglog和hash。这些数据类型都支持push/pop、add/remove及取交集并集和差集及更丰富的操作，同时还具有消息系统的订阅和发布功能，而且这些操作都是原子性的。在此基础上，redis支持各种不同方式的排序。与memcached一样，为了保证效率，数据都是缓存在内存中。区别的是redis会周期性的把更新的数据写入磁盘或者把修改操作写入追加的记录文件，并且在此基础上实现了master-slave(主从)同步。

introduce

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

目录

1.前言

Stream_t.c

关键代码剖析
流ID的结构体
ms代表当前时间
seq为当前序号，这两个字段形成唯一ID

typedef struct streamID {
uint64_t ms; /* Unix time in milliseconds. /
uint64_t seq; / Sequence number. */
} streamID;

每个stream 中含有一个 ID，代表的是这个流中的最后一项的ID
StreamID用来描述流中的对象，而不是流

typedef struct stream {
rax rax; / The radix tree holding the stream. /
uint64_t length; / Number of elements inside this stream. /
streamID last_id; / Zero if there are yet no items. */
rax cgroups; / Consumer groups dictionary: name -> streamCG */ } stream;

streamIterator stream的迭代器

typedef struct streamIterator {
stream stream; / The stream we are iterating. /
streamID master_id; / ID of the master entry at listpack head. /
uint64_t master_fields_count; / Master entries # of fields. */
unsigned char master_fields_start; / Master entries start in listpack. */
unsigned char master_fields_ptr; / Master field to emit next. /
int entry_flags; / Flags of entry we are emitting. /
int rev; / True if iterating end to start (reverse). /
uint64_t start_key[2]; / Start key as 128 bit big endian. /
uint64_t end_key[2]; / End key as 128 bit big endian. /
raxIterator ri; / Rax iterator. */
unsigned char lp; / Current listpack. */
unsigned char lp_ele; / Current listpack cursor. */
unsigned char lp_flags; / Current entry flags pointer. /
/ Buffers used to hold the string of lpGet() when the element is
* integer encoded, so that there is no string representation of the
* element inside the listpack itself. */
unsigned char field_buf[LP_INTBUF_SIZE];
unsigned char value_buf[LP_INTBUF_SIZE]; } streamIterator;

 新建流
 stream *streamNew(void) {
        stream *s = zmalloc(sizeof(*s));
        s->rax = raxNew();
        s->length = 0;
        s->last_id.ms = 0;
        s->last_id.seq = 0;
        s->cgroups = NULL; /* Created on demand to save memory when not used. */
        return s;
    }


void freeStream(stream *s) {
    raxFreeWithCallback(s->rax,(void(*)(void*))lpFree);
    if (s->cgroups)
        raxFreeWithCallback(s->cgroups,(void(*)(void*))streamFreeCG);
    zfree(s);
}