NameNode的启动和停止

版权声明:本文为博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://blog.csdn.net/kaikai_sk/article/details/88686468

1. 安全模式

NameNode启动:
(1)把fsimage加载到内存,已用edits日志。这个过程结束之后,创建一个新的检查点,包括一个新的fsimage和空的edits日志。
(2)监听IPC和HTTP请求。此时只为客户端提供一个只读视图,这种名字节点的只读模式称为安全模式。
(3)获得足够多的DataNode。
(4)当收集到足够多的第二关系信息之后,NameNode会离开安全模式

1.1 安全模式的需求

(1)处于安全模式下,HDFS只提供系统的只读视图,不能进行修改。
(2)NameNode启动时,根据配置,检查第二关系中数据块的副本信息,满足条件时离开安全模式。
(3)通过命令行支持安全模式状态查询、等待和设置。

FSNamesystem.SafeModeInfo

  • 成员变量
 // configuration fields
    /** Safe mode threshold condition %.*/
    private double threshold;
    /** Safe mode extension after the threshold. */
    private int extension;
    /** Min replication required by safe mode. */
    private int safeReplication;
      
    // internal fields
    /** Time when threshold was reached.
     * 
     * <br>-1 safe mode is off
     * <br> 0 safe mode is on, but threshold is not reached yet 
     */
    private long reached = -1;  
    
    /**
    *  用于计算离开安全模式的副本的比例
	**/
    /** Total number of blocks. */
    int blockTotal; 
    /** Number of safe blocks. */
    private int blockSafe;
    
    /** time of the last status printout */
    private long lastStatusReport = 0;
  • blockTotal和blockSafe成员的修改
	 /**
     * Set total number of blocks.
     */
    synchronized void setBlockTotal(int total) 
    {
    	  	this.blockTotal = total; 
	        checkMode();
    }
  /**
     * Increment number of safe blocks if current block has 
     * reached minimal replication.
     * @param replication current replication 
     */
    synchronized void incrementSafeBlockCount(short replication) 
    {
      //达到最低副本水平数
      if ((int)replication == safeReplication)
        this.blockSafe++;
      checkMode();
    }
      
    /**
     * Decrement number of safe blocks if current block has 
     * fallen below minimal replication.
     * @param replication current replication 
     */
    synchronized void decrementSafeBlockCount(short replication) {
      if (replication == safeReplication-1)
        this.blockSafe--;
      checkMode();
    }
  • checkMode
	 /**
     * Check and trigger safe mode if needed. 
     */
    private void checkMode() 
    {
	      if (needEnter()) 
	      {
		        enter();                     // this.reached = 0;
		        reportStatus("STATE* Safe mode ON.", false);
		        return;
	      }
	      // the threshold is reached(已经满足最小副本水平条件)
	      if (!isOn() ||                           // safe mode is off
	          extension <= 0 || threshold <= 0) {      // don't need to wait
		        this.leave(true);                                 // leave safe mode
		        return;
	      }
	      if (reached > 0) {  // threshold has already been reached before(不是第一次满足最小副本水平条件)
	        reportStatus("STATE* Safe mode ON.", false);
	        return;
	      }
	      
	      /**
	      * start monitor
	      * 创建SageModeMonitor对象和它的运行线程
	      **/ 
	      reached = now();
	      smmthread = new Daemon(new SafeModeMonitor());
	      smmthread.start();
	      reportStatus("STATE* Safe mode extension entered.", true);
    }
  • SafeModeMonitor
class SafeModeMonitor implements Runnable 
{
    /** 
	    interval in msec for checking safe mode 
    	检查的时间间隔
    */
    private static final long recheckInterval = 1000;
      
    /**
     */
    public void run() 
    {
      while (fsRunning && (safeMode != null && !safeMode.canLeave())) 
      {
        try 
        {
           //等待1s
          Thread.sleep(recheckInterval);
        }
         catch (InterruptedException ie) 
         {
        }
      }
      // leave safe mode and stop the monitor
      try
     {
        //离开安全模式
        leaveSafeMode(true);
      } 
      catch(SafeModeException es)  //// should never happen
      { 
      String msg = "SafeModeMonitor may not run during distributed upgrade.";
      assert false : msg;
        throw new RuntimeException(msg, es);
      }
      //释放SafeModeMonitor所在的线程
      smmthread = null;
    }
  }
展开阅读全文

启动namenode异常

09-17

查看日志:rnWARN org.apache.hadoop.hdfs.server.namenode.FSEditLog: Unable to determine input streams from QJM to [172.17.138.67:8485, 172.17.138.68:8485, 172.17.138.69:8485]. Skipping.rnorg.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 1 successful responses:rn172.17.138.68:8485: [[786130,786131], [786132,786133], [786134,786135], [786136,786137], [786138,786139]]rn2 exceptions thrown:rn172.17.138.67:8485: Call From EBPPTEST01/172.17.138.67 to EBPPTEST01:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefusedrn172.17.138.69:8485: Call From EBPPTEST01/172.17.138.67 to EBPPTEST03:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefusedrn at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)rn at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:223)rn at org.apache.hadoop.hdfs.qjournal.client.AsyncLoggerSet.waitForWriteQuorum(AsyncLoggerSet.java:141)rn at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.selectInputStreams(QuorumJournalManager.java:459)rn at org.apache.hadoop.hdfs.server.namenode.JournalSet.selectInputStreams(JournalSet.java:250)rn at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1301)rn at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1320)rn at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:613)rn at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:264)rn at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:849)rn at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:609)rn at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:446)rn at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:502)rn at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:658)rn at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:643)rn at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1259)rn at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1325)rnrn看样子是连接拒绝,可是我每台server间的ssh都是互通的 论坛

hadoop 启动异常。jobtracker和namenode启动

03-04

master上jps,结果为rn15740 SecondaryNameNodern15906 Jpsrnrnslave上jps,结果为rn7315 TaskTrackerrn7218 DataNodern7427 Jpsrnrn#start-all.sh执行显示rnstarting namenode, logging to /var/log/hadoop/hadoop/hadoop-hadoop-namenode-vz-121-xinhailong.outrn10.1.1.112: starting datanode, logging to /var/log/hadoop/hadoop/hadoop-hadoop-datanode-localhost.localdomain.outrn10.1.9.121: starting secondarynamenode, logging to /var/log/hadoop/hadoop/hadoop-hadoop-secondarynamenode-vz-121-xinhailong.outrnstarting jobtracker, logging to /var/log/hadoop/hadoop/hadoop-hadoop-jobtracker-vz-121-xinhailong.outrn10.1.1.112: starting tasktracker, logging to /var/log/hadoop/hadoop/hadoop-hadoop-tasktracker-localhost.localdomain.outrnrn#stop-all.sh 显示rnno jobtracker to stoprn10.1.1.112: stopping tasktrackerrnno namenode to stoprn10.1.1.112: stopping datanodern10.1.9.121: stopping secondarynamenodernrn配置文件如下rn#vim /etc/hadoop/core-site.xml rn rn (注意,请先在 hadoopinstall 目录下建立 tmp 文件夹)rn rn hadoop.tmp.dirrn /home/hadoop/project/tmprn A base for other temporary directories.rn rnrn rn fs.default.namern hdfs://ip_master_node:50070rn rnrnrn#vim /etc/hadoop/hdfs-site.xmlrnrn dfs.name.dirrn /home/hadoop/project/hadoop/hdfs/namernrnrn dfs.data.dirrn /home/hadoop/project/hadoop/hdfs/datarnrnrn dfs.replicationrn 1rnrnrn#vim /etc/hadoop/mapred-site.xmlrnrn mapred.job.trackerrn ip_master_node:50030rnrn请问是什么原因导致的? 论坛

wince 启动停止了。

09-28

我在编写流驱动的时候,系统启动突然停止了。rn串口输出信息:rn[dm9: Tries to map io space with 18000000rn[dm9: The mapped address is D0B80000rn[dm9: Chip signature is 90000A46rn[dm9: Chip signature is 90000A46rnrnSTRINGS:***@@****DLL_PROCESS_ATTACH******rnMYSTRINGGS:**@@*******InitKey_Init***********rn*@@**********KeyInt_Setting***********rnrn流驱动部分代码。rnDWORD KEY_Init(DWORD dwContext)rn // if you drive a idiographic device,add oter code ep: int,hardwareinit and so on rn DWORD IDINTThread;rn RETAILMSG(1,(TEXT("MYSTRINGGS:**@@*******InitKey_Init***********\n")));rn //clear the ram which is used for this dllrnrn InitializeAddresses();rn [color=#FF0000]INTInit();[/color]rnrn RETAILMSG(1,(TEXT("@before CreateThread \r\n")));rn Key_Key_Thread = CreateThread(0,0,(LPTHREAD_START_ROUTINE)Key_KeyThread_Fun,0,0,&IDINTThread);rn if(!Key_Key_Thread)rn rn RETAILMSG(1,(TEXT("@CreateThread faile \r\n")));rn return FALSE;rn rn elsern rn RETAILMSG(1,(TEXT("@CreateThread OK \r\n")));rn rn Key_KeyIntr_Event = CreateEvent(NULL,false,false,NULL);rn if(!Key_KeyIntr_Event)rn rn RETAILMSG(1,(TEXT("@CreateEvent faile \r\n")));rn return FALSE;rn rn elsern rn RETAILMSG(1,(TEXT("@CreateEvent OK \r\n")));rn rn。。。。。。。。rn。。。。。。。rnrn[color=#FF0000]BOOL INTInit()[/color]rnrnrn RETAILMSG(1,(TEXT("*@@**********KeyInt_Setting***********\r\n")));rnrn v_pIOPregs->GPFCON &= ~(0x3 << 2); /* Set EINT1(GPF1) as EINT1 */rn v_pIOPregs->GPFCON |= (0x2 << 2);rnrn v_pIOPregs->EXTINT1 &= ~(0x7 << 3);rn v_pIOPregs->EXTINT1 |= (0x2 << 3); /* Configure EINT1 as Falling Edge Mode */rnrn return TRUE;rnrn初学,不知道哪里出问题了。帮忙分析下。 论坛

没有更多推荐了,返回首页