Hadoop NameNode格式化

   Namenode格式化是由NameNode FSImage Configuration三个类共同作用完成的，在执行格式化前由Configuration和Namenode两个类载入相应的配置文件，在静态代码段完成。

NameNode.java
static{
    Configuration.addDefaultResource("hdfs-default.xml");
    Configuration.addDefaultResource("hdfs-site.xml");
  }
 
Configuration.java:
static{
    //print deprecation warning if hadoop-site.xmlis found in classpath
    ClassLoader cL = Thread.currentThread().getContextClassLoader();
    if (cL == null) {
      cL = Configuration.class.getClassLoader();
    }
    if(cL.getResource("hadoop-site.xml")!=null){
      LOG.warn("DEPRECATED:hadoop-site.xml found in the classpath.
 "+
          "Usage of hadoop-site.xml isdeprecated. Instead use
 core-site.xml, "
          + "mapred-site.xml and hdfs-site.xmlto override properties
of " +
          "core-default.xml,mapred-default.xml and hdfs-default.xml
" + "respectively");
    }
    addDefaultResource("core-default.xml");
    addDefaultResource("core-site.xml");
  }

可以看到在这个过程中需要加载的配置文件有4个

hdfs-default.xml

hdfs-site.xml

core-default.xml

core-site.xml

格式化是通过执行NameNode类的main函数来实现，主要通过下面函数完成格式化：

NameNode namenode = createNameNode(argv, null);

createNameNode的执行流程：

    获取命令行参数，进入switch判断，然后执行NameNode.format函数，进入该函数后要从配置文件中获取相应目录1、fsimage文件目录 2、edits目录。然后判断是否为重新格式化，如果是则等待用户确认，然后进入下面流程：

FSNamesystem nsys =
        new FSNamesystem(new FSImage(dirsToFormat,editDirsToFormat),conf);
nsys.dir.fsImage.format();

 

真正执行格式话的任务由FSImage.saveFSImage实现：

/*
  * Save the contents of the FS image to thefile.
   */
void saveFSImage(FilenewFile) throws IOException {
    FSNamesystem fsNamesys = FSNamesystem.getFSNamesystem();
    FSDirectory fsDir = fsNamesys.dir;
    long startTime = FSNamesystem.now();
    //
    // Write out data
    //
DataOutputStream out =
NewDataOutputStream(new BufferedOutputStream(
                     newFileOutputStream(newFile)));
    try {
      out.writeInt(FSConstants.LAYOUT_VERSION);
      out.writeInt(namespaceID);
      out.writeLong(fsDir.rootDir.numItemsInTree());
     out.writeLong(fsNamesys.getGenerationStamp());
      byte[] byteStore = new
byte[4*FSConstants.MAX_PATH_LENGTH];
      ByteBuffer strbuf = ByteBuffer.wrap(byteStore);
      // save the root
      saveINode2Image(strbuf, fsDir.rootDir,out);
      // save the rest of the nodes
      saveImage(strbuf, 0, fsDir.rootDir, out);
     fsNamesys.saveFilesUnderConstruction(out);
      fsNamesys.saveSecretManagerState(out);
      strbuf = null;
    } finally {
      out.close();
}