MESI protocol and MOESI protocol

 
MESI

MESI protocol - Wikipedia

States

Modified (M)

The cache line is present only in the current cache, and is dirty - it has been modified (M state) from the value in main memory. The cache is required to write the data back to main memory at some time in the future, before permitting any other read of the (no longer valid) main memory state. The write-back changes the line to the Shared state(S).

Exclusive (E)

The cache line is present only in the current cache, but is clean - it matches main memory. It may be changed to the Shared state at any time, in response to a read request. Alternatively, it may be changed to the Modified state when writing to it.

Shared (S)

Indicates that this cache line may be stored in other caches of the machine and is clean - it matches the main memory. The line may be discarded (changed to the Invalid state) at any time. 

注: shared状态时cache line may be in other cahches？是由于other caches evict该cache line时，造成的？coherence bus应该可以避免这种可能，因bus可以详细记录各cache line copy的状态。

注：ncore是否也有may be这种情况？？？

Invalid (I)

Indicates that this cache line is invalid (unused).

Operation

The MESI protocol is defined by a finite-state machine that transitions from one state to another based on 2 stimuli.

The first stimulus is the processor specific Read and Write request. For example: A processor P1 has a Block X in its Cache, and there is a request from the processor to read or write from that block.

The second stimulus is given through the bus connecting the processors. In particular the "Bus side requests" come from other processors that don't have the cache block or the updated data in their Cache. The bus requests are monitored with the help of Snoopers,[4] which monitor all the bus transactions.

Following are the different type of Processor requests and Bus side requests:

Processor Requests to Cache include the following operations:

1. PrRd: The processor requests to read a Cache block.
2. PrWr: The processor requests to write a Cache block

Bus side requests are the following:

1. BusRd: Snooped request that indicates there is a read request to a Cache block requested by another processor
2. BusRdX: Snooped request that indicates there is a write request to a Cache block requested by another processor that doesn't already have the block.  (注：snoop当前processor，获取并invalid该cache line copy)
3. BusUpgr: Snooped request that indicates that there is a write request to a Cache block requested by another processor that already has that cache block residing in its own cache. (注：snooop当前processor，invalid该cache line copy)
4. Flush: Snooped request that indicates that an entire cache block is written back to the main memory by another processor.
5. FlushOpt: Snooped request that indicates that an entire cache block is posted on the bus in order to supply it to another processor (Cache to Cache transfers).

(Such Cache to Cache transfers can reduce the read miss latency if the latency to bring the block from the main memory is more than from Cache to Cache transfers, which is generally the case in bus based systems.)

Snooping Operation: In a snooping system, all caches on a bus monitor all the transactions on that bus. Every cache has a copy of the sharing status of every block of physical memory it has stored. The state of the block is changed according to the State Diagram of the protocol used. (Refer image above for MESI state diagram). The bus has snoopers on both sides:

1. Snooper towards the Processor/Cache side.
2. The snooping function on the memory side is done by the Memory controller.
3. 注：总线上snooper通常只存在一处，与该处描述不符

Table 1.1 State Transitions and response to various Processor Operations

Initial State	Operation	Response
Invalid(I)	PrRd	Issue BusRd to the bus other Caches see BusRd and check if they have a valid copy, inform sending cache State transition to (S)Shared, if other Caches have valid copy. State transition to (E)Exclusive, if none (must ensure all others have reported). If other Caches have copy, one of them sends value, else fetch from Main Memory
	PrWr	Issue BusRdX signal on the bus State transition to (M)Modified in the requestor Cache. If other Caches have copy, they send value, otherwise fetch from Main Memory If other Caches have copy, they see BusRdX signal and invalidate their copies. Write into Cache block modifies the value. 注：如直接写某个bytes, 2 bytes, 4 bytes等，不足cache line size的写操作!
Exclusive(E)	PrRd	No bus transactions generated State remains the same. Read to the block is a Cache Hit
	PrWr	No bus transaction generated State transition from Exclusive to (M)Modified Write to the block is a Cache Hit
Shared(S)	PrRd	No bus transactions generated State remains the same. Read to the block is a Cache Hit.
	PrWr	Issues BusUpgr signal on the bus. State transition to (M)Modified. other Caches see BusUpgr and mark their copies of the block as (I)Invalid.
Modified(M)	PrRd	No bus transactions generated State remains the same. Read to the block is a Cache hit
	PrWr	No bus transactions generated State remains the same. Write to the block is a Cache hit.

Table 1.2 State Transitions and response to various Bus Operations

Initial State	Operation	Response
Invalid(I)	BusRd	No State change. Signal Ignored.
	BusRdX/BusUpgr	No State change. Signal Ignored
Exclusive(E)	BusRd	Transition to Shared (Since it implies a read taking place in other cache). Put FlushOpt on bus together with contents of block.
	BusRdX	Transition to Invalid. Put FlushOpt on Bus, together with the data from now-invalidated block.
Shared(S)	BusRd	No State change (other cache performed read on this block, so still shared). May put FlushOpt on bus together with contents of block (design choice, which cache with Shared state does this). (注:由哪个cache copy来发送数据由设计决定！)
	BusRdX/BusUpgr	Transition to Invalid (cache that sent BuxRdX/BusUpgr becomes Modified) May put FlushOpt on bus together with contents of block (design choice, which cache with Shared state does this). (注:BusUpgr不需要发送数据)
Modified(M)	BusRd	Transition to (S)Shared. Put FlushOpt on Bus with data. Received by sender of BusRd and Memory Controller, which writes to Main memory. (注：把数据发送给发起snoop的Master，并回写Main memory)
	BusRdX	Transition to (I)Invalid. Put FlushOpt on Bus with data. Received by sender of BusRdx and Memory Controller, which writes to Main memory. (注：把数据发送给发起snoop的Master，并回写Main memory) 注1：ncore的IO写 cpu modified parial cache line对应地址时，dirty data是否也是如此操作？即dirty data返回到IO buffer测，同时回写至Main memory，后在buffer执行RMW(partial)操作后，再次回写至Main memory？此处总线可以优化!  注2：此处是否可以不回写memory？ Inter Xeon CPU是支持cache to cache的优化的，参考第张一图和最后张图  CPU缓存一致性保障原理 - 知乎 (zhihu.com)  注3：ncore的IO写 cpu modified entry cache line对应地址时，ncore表现应该是，invalid cpu cache line，IO直接写Main memory

1，The Modified and Exclusive states are always precise: i.e. they match the true cache line ownership situation in the system. The Shared state may be imprecise: if another cache discards a Shared line, this cache may become the sole owner of that cache line, but it will not be promoted to Exclusive state.

2，A write may only be performed freely if the cache line is in the Modified or Exclusive state. If it is in the Shared state, all other cached copies must be invalidated first. This is typically done by a broadcast operation known as Request For Ownership (RFO).

3，两个shared状态的cache，同时对同一cacheline发起BusUpgr；或两个invalid状态的cache，同时对同一cacheline发起BusRdX；详细处理过程？ 应该是交由interconnect仲裁处理！interconnect如何响应被拒绝的chache(cpu)？

4，MESI协议状态转换在线网站： VivioJS MESI help (tcd.ie) 

MOESI

MOESI protocol - Wikipedia

In computing, MOESI is a full cache coherency protocol that encompasses all of the possible states commonly used in other protocols. In addition to the four common MESI protocol states, there is a fifth "Owned" state representing data that is both modified and shared. This avoids the need to write modified data back to main memory before sharing it. While the data must still be written back eventually, the write-back may be deferred.

In order for this to be possible, direct cache-to-cache transfers of data must be possible, so a cache with the data in the modified state can supply that data to another reader without transferring it to memory.

(注：MESI也可以实现不回写至memory，直接交由另一cache操作后，回写至memory；区别在于当前cache交出去后，需invalid)

Modified

This cache has the only valid copy of the cache line, and has made changes to that copy. The cached copy may be further modified freely.

Owned

This line is one of several copies in the system. This cache does not have permission to modify the copy but the line is modified (dirty) relative to main memory, and this cache has the exclusive responsibility for ensuring main memory is eventually updated. The cache line may be changed to the Modified state after invalidating all shared copies, or changed to the Shared state by writing the modifications back to main memory. Owned cache lines must respond to a snoop request with data, to ensure the stale copy in main memory is not used.

注：Owned应为Modified状态的cache line copy，被其他cache读取后，转换得到的状态。

Exclusive

This cache has the only copy of the line, but the line is clean (unmodified). It may be written to, changing to the Modified state.

Shared

This line is one of several copies in the system. This cache does not have permission to modify the copy. Unlike the MESI protocol, a shared cache line may be dirty with respect to memory; if it is, one cache has a copy in the Owned state, and that cache is responsible for eventually updating main memory. If no cache holds the line in the Owned state, the memory copy is up to date. The cache line may not be written, but must be changed to the Exclusive or Modified state first, by invalidating all other cached copies. (If the cache line was Owned before, the invalidate response will indicate this, and the state will become Modified, so the obligation to eventually write the data back to memory is not forgotten.) It may also be discarded (changed to the Invalid state) at any time.

注：shared状态的cache line与owner状态的cache line一致，clean或dirty。

Invalid

This block is not valid; it must be fetched to satisfy any attempted access.

This protocol, a more elaborate version of the simpler MESI protocol, avoids the need to write a dirty cache line back to main memory when another processor tries to read it. Instead, the Owned state allows a processor to supply the modified data directly to the other processor. This is beneficial when the communication between two CPUs is significantly better than to main memory. An example would be multi-core CPUs with per-core L2 caches.

While MOESI can quickly share dirty cache lines from cache, it may struggle to quickly share clean lines from cache. If a cache line is clean with respect to memory and in the shared state, then there is no obvious single candidate cache to respond to a read request, so it is normal to let the read request be filled from memory. (This is solved by the MESIF protocol, which may be combined with MOESI to make MOESIF.)

注：系统当前只有shared状态的cache line，没有owner，无法确定由谁来响应读数据。不合理，MESI都能解决，MOESI也不应存在该问题。且即使如此总线也能解决！

If a processor wishes to write to an Owned cache line, it must notify the other processors which are sharing that cache line. The standard implementation simply tells them to invalidate their copies, moving its own copy to the Modified state when this is complete, but alternatively it may use a write-through policy, telling them to update their copies with the new contents. This is a partial write-through which does not go as far as main memory; the processor's own copy remains in the Owned state.

ARMv8-缓存一致性（cache coherency）解决方案：MOESI protocol_moesi协议-CSDN博客

上述链接所示的MOESI状态图，是不完整的实现方案；不含 Shared -> Owned、不含Owned -> Shared 两条转换路径。