rabbitmq中消息的存储

最新推荐文章于 2024-04-04 08:08:06 发布

qq_26182553

最新推荐文章于 2024-04-04 08:08:06 发布

阅读量1.6k

点赞数

分类专栏：消息队列

消息队列专栏收录该内容

7 篇文章 0 订阅

订阅专栏

转自：https://blog.csdn.net/yongche_shi/article/details/51500534

1. 大概原理：

所有队列中的消息都以append的方式写到一个文件中，当这个文件的大小超过指定的限制大小后，关闭这个文件再创建一个新的文件供消息的写入。文件名（*.rdq）从0开始然后依次累加。当某个消息被删除时，并不立即从文件中删除相关信息，而是做一些记录，当垃圾数据达到一定比例时，启动垃圾回收处理，将逻辑相邻的文件中的数据合并到一个文件中。

2. 消息的读写及删除：

rabbitmq在启动时会创建msg_store_persistent,msg_store_transient两个进程，一个用于持久消息的存储，一个用于内存不够时，将存储在内存中的非持久化数据转存到磁盘中。所有队列的消息的写入和删除最终都由这两个进程负责处理，而消息的读取则可能是队列本身直接打开文件进行读取，也可能是发送请求由msg_store_persisteng/msg_store_transient进程进行处理。

在进行消息的存储时，rabbitmq会在ets表中记录消息在文件中的映射，以及文件的相关信息。消息读取时，根据消息ID找到该消息所存储的文件，在文件中的偏移量，然后打开文件进行读取。消息的删除只是从ets表删除指定消息的相关信息，同时更新消息对应存储的文件的相关信息（更新文件有效数据大小）。

 
   [java]  
   view plaincopy
<code class="hljs erlang" style="padding:.5em;margin:0px;display:block;color:rgb(101,123,131);background:rgb(253,246,227);"><span class="hljs-pp" style="padding:0px;margin:0px;"><span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">-record</span><span class="hljs-params" style="padding:0px;margin:0px;">(msg_location, <span class="hljs-tuple" style="padding:0px;margin:0px;">{ msg_id,     <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%消息ID</span>  
                        ref_count,  <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%引用计数</span>  
                        file,       <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%消息存储的文件名</span>  
                        offset,     <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%消息在文件中的偏移量</span>  
                        total_size  <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%消息的大小</span>  
                      }</span>)</span></span>.  
  
<span class="hljs-pp" style="padding:0px;margin:0px;"><span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">-record</span><span class="hljs-params" style="padding:0px;margin:0px;">(file_summary, <span class="hljs-tuple" style="padding:0px;margin:0px;">{ file,       <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%文件名</span>  
                        valid_total_size, <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%文件有效数据大小</span>  
                        left,       <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%位于该文件左边的文件</span>  
                        right,      <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%位于该文件右边的文件</span>  
                        file_size,  <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%文件总的大小</span>  
                        locked,     <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%上锁标记 垃圾回收时防止对文件进行操作</span>  
                        readers     <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">%%当前读文件的队列数</span>  
                      }</span>)</span></span></code>  

3. 垃圾回收：

由于执行消息删除操作时，并不立即对在文件中对消息进行删除，也就是说消息依然在文件中，仅仅是垃圾数据而已。当垃圾数据超过一定比例后（默认比例为50%），并且至少有三个及以上的文件时，rabbitmq触发垃圾回收。垃圾回收会先找到符合要求的两个文件（根据#file_summary{}中left，right找逻辑上相邻的两个文件，并且两个文件的有效数据可在一个文件中存储），然后锁定这两个文件，并先对左边文件的有效数据进行整理，再将右边文件的有效数据写入到左边文件，同时更新消息的相关信息（存储的文件，文件中的偏移量），文件的相关信息（文件的有效数据，左边文件，右边文件），最后将右边的文件删除。

4. 性能考虑：

（1）操作引用计数（flying_ets）

队列在进行消息的写入和删除操作前，会在flying_ets表里通过+1，-1的方式进行计数，然后投递请求给msg_store_persistent/msg_store_transient进程进行处理，进程在真正写操作或者删除之前会再次判断flying_ets中对应消息的计数决定是否需要进行相应操作。这样，对于频繁写入和删除的操作，概率减少实际的写入和删除。

 
   [java]  
   view plaincopy
<code class="hljs r" style="padding:.5em;margin:0px;display:block;color:rgb(101,123,131);background:rgb(253,246,227);">client_write(MsgId, Msg, Flow,  
             CState=<span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">#client_msstate{cur_file_cache_ets=CurFileCacheEts,</span>  
                                    client_ref=CRef}) ->  
    ok = client_update_flying(+<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">1</span>, MsgId, CState),  
    ok = update_msg_cache(CurFileCacheEts, MsgId, Msg),  
    ok = server_cast(CState, {write, CRef, MsgId, Flow}).  
  
remove(MsgIds, CState = <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">#client_msstate { client_ref = CRef }) -></span>  
    [client_update_flying(-<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">1</span>, MsgId, CState) || MsgId <- MsgIds],  
    server_cast(CState, {remove, CRef, MsgIds}).  
  
client_update_flying(Diff, MsgId,  
                     <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">#client_msstate{flying_ets = FlyingEts,</span>  
                                     client_ref = CRef}) ->  
    Key = {MsgId, CRef},  
    case ets:insert_new(FlyingEts, {Key, Diff}) of  
        true  ->  
            ok;  
        false ->  
            <span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">try</span> ets:update_counter(FlyingEts, Key, {<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">2</span>, Diff}) of  
            <span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">...</span>  
    end.  
  
handle_cast({write, CRef, MsgId, Flow},  
            State = <span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">#msstate{cur_file_cache_ets=CurFileCacheEts,</span>  
                             clients=Clients}) ->  
    <span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">...</span>  
    true = <span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">0</span> =< ets:update_counter(CurFileCacheEts, MsgId, {<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">3</span>, -<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">1</span>}),  
    case update_flying(-<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">1</span>, MsgId, CRef, State) of  
        process ->  
            [{MsgId,Msg,_PWC}]=ets:lookup(CurFileCacheEts, MsgId),  
            noreply(write_message(MsgId, Msg, CRef, State));  
        ignore ->  
            <span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">...</span>  
    end;  
  
handle_cast({remove, CRef, MsgIds}, State) ->  
    {RemovedMsgIds, State1} =  
        lists:foldl(  
            fun (MsgId, {Removed, State2}) ->  
                case update_flying(+<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">1</span>, MsgId, CRef, State2) of  
                    process ->  
                        {[MsgId | Removed],  
                        remove_message(MsgId, CRef, State2)};  
                    ignore ->  
                        {Removed, State2}  
                end  
            end, {[], State}, MsgIds),  
    <span class="hljs-keyword" style="padding:0px;margin:0px;color:rgb(133,153,0);">...</span>  
  
update_flying(Diff,MsgId,CRef,<span class="hljs-comment" style="padding:0px;margin:0px;color:rgb(147,161,161);">#msstate{flying_ets = FlyingEts }) -></span>  
    Key = {MsgId, CRef},  
    NDiff = -Diff,  
    case ets:lookup(FlyingEts, Key) of  
        [] ->  
            ignore;  
        [{_,  Diff}] ->  
            ignore;  
        [{_, NDiff}] ->  
            ets:update_counter(FlyingEts, Key, {<span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">2</span>, Diff}),  
            true = ets:delete_object(FlyingEts, {Key, <span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">0</span>}),  
            process;  
        [{_, <span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">0</span>}] ->  
            true = ets:delete_object(FlyingEts, {Key, <span class="hljs-number" style="padding:0px;margin:0px;color:rgb(42,161,152);">0</span>}),  
            ignore;  
        [{_, Err}] ->  
            throw({bad_flying_ets_record, Diff, Err, Key})  
    end.</code>  

（2）尽可能的并发读

在读取消息的时候，都先根据消息ID找到对应存储的文件，如果文件存在并且未被锁住，则直接打开文件，从指定位置读取消息的内容。

如果消息存储的文件被锁住了，或者对应的文件不存在了，则发送请求，由msg_store_persistent/msg_store_transient进程进行处理。

（3）消息缓存

1）利用ets表进行缓存

对于当前正在写的文件，所有消息在写入前都会在cur_file_cache_ets表中存一份，消息读取时会优先从这里进行查找。文件关闭时，会将cur_file_cache_ets表中引用计数为0的消息进行清除。

2）file_handle_cache的写缓存

rabbitmq中对文件的操作封转到了file_handle_cache模块，以写模式打开文件时，默认有1M大小的缓存，即在进行文件的写操作时，是先写入到这个缓存中，当缓存超过大小或者显式刷新，才将缓存中的内容刷入磁盘中。

 
   [java]  
   view plaincopy
<code class="hljs handlebars" style="padding:.5em;margin:0px;display:block;color:rgb(101,123,131);background:rgb(253,246,227);"><span class="xml" style="padding:0px;margin:0px;">rabbit_msg_store.erl  
  
-define(HANDLE_CACHE_BUFFER_SIZE, 1048576). %% 1MB  
  
open_file(Dir, FileName, Mode) ->  
    file_handle_cache:open(form_filename(Dir, FileName),  
                           ?BINARY_MODE ++ Mode,  
                           [{write_buffer,?HANDLE_CACHE_BUFFER_SIZE}]).  
  
file_handle_cache.erl  
  
append(Ref,Data) ->  
    with_handles(  
        [Ref],  
        fun ([#handle { is_write = false }]) ->  
           {error, not_open_for_writing};  
        ([Handle]) ->  
            case maybe_seek(eof, Handle) of  
                </span><span class="hljs-expression" style="padding:0px;margin:0px;">{{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">ok</span>, _<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Offset</span>}, <span class="hljs-begin-block" style="padding:0px;margin:0px;">#handle</span>{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">hdl</span> = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Hdl</span>,  
                                        <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">offset</span> = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Offset</span>,  
                                        <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">size</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">limit</span> = 0,  
                                        <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">at</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">eof</span> = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">true</span> }= <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>1} <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">-</span>>  
                    <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Offset</span>1 = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Offset</span> + <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">iolist</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">size</span>(<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Data</span>),  
                    {<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">prim</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">file</span>:<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>(<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Hdl</span>, <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Data</span>),  
                    [<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>1<span class="hljs-begin-block" style="padding:0px;margin:0px;">#handle</span>{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">is</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">dirty</span>=<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">true</span>,<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">offset</span>=<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Offset</span>1 }]};  
                {{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">ok</span>, _<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Offset</span>},<span class="hljs-begin-block" style="padding:0px;margin:0px;">#handle</span>{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span> = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">WriteBuffer</span>,  
                                       <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">size</span> = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Size</span>,  
                                       <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">size</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">limit</span>= <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Limit</span>,  
                                       <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">at</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">eof</span> = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">true</span> } = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>1} <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">-</span>>  
                    <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">WriteBuffer</span>1 = [<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Data</span> | <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">WriteBuffer</span>],  
                    <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Size</span>1 = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Size</span> + <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">iolist</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">size</span>(<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Data</span>),  
                    <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>2=<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>1<span class="hljs-begin-block" style="padding:0px;margin:0px;">#handle</span>{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span>=<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">WriteBuffer</span>1,  
                                           <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">size</span>=<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Size</span>1},  
                    <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">case</span> <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Limit</span> =/= <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">infinity</span> <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">andalso</span> <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Size</span>1 > <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Limit</span> <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">of</span>  
                        <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">true</span>  <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">-</span>>  
                            {<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Result</span>,<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>3} = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">write</span>_<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">buffer</span>(<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>2),  
                            {<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Result</span>, [<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>3]};  
                        <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">false</span> <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">-</span>>  
                            {<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">ok</span>, [<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>2]}  
                    <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">end</span>;  
                {{<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">error</span>, _} = <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Error</span>, <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>1} <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">-</span>>  
                    {<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Error</span>, [<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">Handle</span>1]}  
            <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">end</span>  
        <span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">end</span>)<span class="hljs-variable" style="padding:0px;margin:0px;color:rgb(181,137,0);">.</span></span></code>  