SeaWeedfs学习总结
SeaWeedfs最初是作为一个对象存储来有效地处理小文件的。central master不管理central master中的所有文件元数据,而只管理文件卷,它通过这些volume servers管理文件及其元数据。这减轻了来自central master的并发压力,并将文件元数据分散到卷服务器中,从而允许更快的文件访问(O(1),通常只有一个磁盘读取操作)。每个文件的元数据只有40字节的磁盘存储开销。
1 架构
官网描述:
Usually distributed file systems split each file into chunks, a central master keeps a mapping of filenames, chunk indices to chunk handles, and also which chunks each chunk server has.
The main drawback is that the central master can’t handle many small files efficiently, and since all read requests need to go through the chunk master, so it might not scale well for many concurrent users.
Instead of managing chunks, SeaweedFS manages data volumes in the master server. Each data volume is 32GB in size, and can hold a lot of files. And each storage node can have many data volumes. So the master node only needs to store the metadata about the volumes, which is a fairly small amount of data and is generally stable…
seaWeedFs的文件存储的基本结构如上,我们从下往上看
数据卷:也就是是类似我们windowspan的D,E盘等物理磁盘,文件保存的物理介质。默认32GB,通过修改代码可以修改为64GB或者128GB,每个文件的最大不超过单个卷的大小。
官网描述:In the current implementation, each volume can hold 32 gibibytes (32GiB or 8x2^32 bytes). This is because we align content to 8 bytes. We can easily increase this to 64GiB, or 128GiB, or more, by changing 2 lines of code, at the cost of some wasted padding space due to alignment.
There can be 4 gibibytes (4GiB or 2^32 bytes) of volumes. So the total system size is 8 x 4GiB x 4GiB which is 128 exbibytes (128EiB or 2^67 bytes).
Each individual file size is limited to the volume size.
数据卷服务器&