hadoop知识之fsimage和editlog

fsimage：filesystem image 的简写，文件镜像。

客户端修改文件时候，先写到editlog，成功后才更新内存中的metadata信息。

只有当对文件操作成功的时候，才会写到editlog。

fsimage是文件meta信息的持久化的检查点。

当namenode失败的时候，

文件metadata信息可以通过：加载fsimage文件到内存中，在editlog中应用相应的操作。

(这一操作也是namenode刚启动要干的事情。)

fsimage文件中包含文件metadata信息，不包含文件块位置的信息。

namenode把文件块位置信息存储在内存中，这些位置信息是，在 datanode加入集群的时候，

namenode询问datanode得到的，并且间断的更新。

As described, the edits file would grow without bound. Though this state of affairs
would have no impact on the system while the namenode is running, if the namenode
were restarted, it would take a long time to apply each of the operations in its (very
long) edit log. During this time, the filesystem would be offline, which is generally
undesirable.

The solution is to run the secondary namenode, whose purpose is to produce checkpoints
of the primary’s in-memory filesystem metadata

fsimage与editlog的旅行

图片

 

两个属性配置checkpoint

fs.checkpoint.period in seconds

fs.checkpoint.size in bytes, which it checks
every five minutes.

safe mode下也可以手动,

命令： hadoop dfsadmin -saveNamespace