Week 3: 边下边播完整性校验作业

边下边播完整性校验

week 3最后的编程作业是解决体积较大的视频文件边下边播完整性校验问题, 原题如下:

Suppose a web site hosts large video file $F$ that anyone can download. Browsers who download the file need to make sure the file is authentic before displaying the content to the user. One approach is to have the web site hash the contents of $F$ using a collision resistant hash and then distribute the resulting short hash value $h=H(F)$ to users via some authenticated channel (later on we will use digital signatures for this). Browsers would download the entire file FF, check that $H(F)$ is equal to the authentic hash value $h$ and if so, display the video to the user. Unfortunately, this means that the video will only begin playing after the entire file $F$ has been downloaded.

Our goal in this project is to build a file authentication system that lets browsers authenticate and play video chunks as they are downloaded without having to wait for the entire file. Instead of computing a hash of the entire file, the web site breaks the file into 1KB blocks (1024 bytes). It computes the hash of the last block and appends the value to the second to last block. It then computes the hash of this augmented second to last block and appends the resulting hash to the third block from the end. This process continues from the last block to the first as in the following diagram:

The final hash value $h_0–a$ hash of the first block with its appended hash – is distributed to users via the authenticated channel as above.

Now, a browser downloads the file $F$ one block at a time, where each block includes the appended hash value from the diagram above. When the first block $(B_0||h_1)$ is received the browser checks that $H(B_0||h_1)$ is equal to $h_0$ and if so it begins playing the first video block. When the second block
$(B_1||h_2)$ is received the browser checks that $H(B_1||h_2)$ is equal to $h_1$ and if so it plays this second block. This process continues until the very last block. This way each block is authenticated and played as it is received and there is no need to wait until the entire file is downloaded.

It is not difficult to argue that if the hash function $H$ is collision resistant then an attacker cannot modify any of the video blocks without being detected by the browser. Indeed, since $h_0=H(B_0||h_1)$ an attacker cannot find a pair $(B_0^{{}^{\prime }},h_1^{{}^{\prime }})\ne (B_0,h_1)$ such that $h_0=H(B_0^{{}^{\prime }},h_1^{{}^{\prime }})$ since this would break collision resistance of $H$. Therefore after the first hash check the browser is convinced that both $B_0$ and $h_1$ are authentic. Exactly the same argument proves that after the second hash check the browser is convinced that both $B_1$ and $h_2$ are authentic, and so on for the remaining blocks.

In this project we will be using SHA256 as the hash function. For an implementation of SHA256 use an existing crypto library such as PyCrypto (Python), Crypto++ (C++), or any other.

When appending the hash value to each block, please append it as binary data, that is, as 32 unencoded bytes (which is 256 bits). If the file size is not a multiple of 1KB then the very last block will be shorter than 1KB, but all other blocks will be exactly 1KB.
Your task is to write code to compute the hash $h_0$ of a given file $F$ and to verify blocks of $F$ as they are received by the client.
In the box below please enter the (hex encoded) hash $h_0$ for this video file.
You can check your code by using it to hash a different file. In particular, the hex encoded $h_0$ for this video file is:
03c08f4ee0b576fe319338139c045c89c3e8e9409633bea29442e21425006ea8

解题思路

如题中所示，分为两部分：

1. 将文件按1KB(1024B)大小进行分块
2. 然后逆序从最后一块往前算sha256
3. 直到算出第首块的hash值$h_0$

样例程序

代码

from Crypto.Hash import SHA256
from typing import Tuple, List


def split_file_into_chunks(path: str, chunk: int) -> List[bytes]:
    """
    将文件按大小切分成块
    :param path: 文件路径
    :param chunk: 大小
    :return:
    """
    with open(path, 'rb') as ivl:
        bytes_content = ivl.read()
        total_len = len(bytes_content)
        chunks = list()
        for i in range(0, total_len, chunk):
            chunks.append(bytes_content[i: i + chunk])
        return chunks

def compute_h0_hash(chunks: List[bytes]) -> (SHA256.SHA256Hash, List[Tuple[bytes, str]]):
    """
    计算hash
    :param chunks:
    :return:
    """
    result_list = list()
    last_chuck_hash = SHA256.new(chunks[-1])
    for i in range(len(chunks) - 2, -1, -1):
        result_list.append((chunks[i], last_chuck_hash))
        last_chuck_hash = SHA256.new(chunks[i] + last_chuck_hash.digest())
    result_list.reverse()  # 翻转，使得第一块在最前面
    return last_chuck_hash, result_list


if __name__ == "__main__":
    chunks = split_file_into_chunks("6.2.birthday.mp4_download", 1024)
    h0, _ = compute_h0_hash(chunks)
    print("test h0: {}".format(h0.hexdigest()))

    chunks = split_file_into_chunks("6.1.intro.mp4_download", 1024)
    h0, _ = compute_h0_hash(chunks)
    print("trail h0: {}".format(h0.hexdigest()))

结果