Txilm Protocol: Monoxide公链系统中的区块压缩算法

今天介绍一下将在Monoxide公链系统中采用的区块压缩算法。从比特币的BIP152出发的进一步改进算法。使用Txilm 算法将使的区块传输带宽需求降低到原来的1/80，同时使得冲突概率控制在1/1000左右，消解冲突的计算量可以忽略不计。这个算法不依赖各个全节点之间的mempool是高度一致的，无需额外的协议保持mempool的同步。

Txilm Protocol: Lossy Block Compression with Salted Short Hashing

Background

Compact blocks carries only TXIDs by assuming most TXes in a newly created block are already stored in the mempool for most full nodes. This is proposed in BIP152. More details can be found inbitcoincore.org/en/2016

Compact blocks basically replace each TX (300–400 bytes roughly) with a 32-bytes TXID (e.g. SHA256 of the TX). This yields a nearly 10 times bandwidth saving.

We aim to further reduce the size of each TX representation in a compact block to around 40 bits. We can achieve 6.4x compression over the original proposal of Compact blocks (6.4 = 32 bytes / 40 bits). The new proposal is simply without using additional data structures like bloom filter or IBLT. Also, the new proposal doesn’t rely on consistent mempool across full nodes.

Combined with Canonical transaction ordering rule (CTOR), the short hashing can be further reduced to 32 bits, which yields 8x compression over the original proposal of Compact blocks. A totally 80 times data size reduction is realized.

Rationale

We present each TX in a block by a small hash value based on TXID:

TXID-HASH = h(TXID)

In which h is a small hash function that generates 32-bit to 64-bit hash values. It can be just CRC32, CRC40 or CRC64. The proposed new compact block scheme includes only a list of TXID_HASH, ordered as the original list of TX.

Ambiguity may occurs with such a k-bit small-sized hash, which needs to be resolved by each full node. Once receiving a new block that includes the TXID-HASH list from the sender, the receiver searches each received TXID-HASH in the hash list produced by its mempool. For each TXID_HA