hadoopd的secondarynamenode的使用

最新推荐文章于 2021-03-03 20:06:37 发布
最新推荐文章于 2021-03-03 20:06:37 发布
阅读量74 收藏
点赞数
文章标签： 大数据

本文参考：http://www.cnblogs.com/ggjucheng/archive/2012/04/18/2454693.html，本文使用的是伪分布式的hadoop集群，版本为1.0.4，主要的配置文件如下：

core-site.xml:

<!-- Put site-specific property overrides in this file. -->
<configuration>
	<property>
		<name>hadoop.tmp.dir</name>
		<value>/home/hadoop/fs/tmp</value>
	</property>
	<property>
		<name>fs.default.name</name>
		<value>hdfs://hadoop:9000</value>
	</property>
	<property>
		<name>fs.checkpoint.period</name>
		<value>300</value>
	</property>
	<property>
		<name>fs.checkpoint.size</name>
		<value>67108864</value>
	</property>
	<property>
		<name>fs.checkpoint.dir</name>
		<value>/home/hadoop/checkpoint/namesecodary</value>
	</property>
</configuration>

hdfs-site.xml:

<!-- Put site-specific property overrides in this file. -->
<configuration>
	<property>
		<name>dfs.replicatioin</name>
		<value>1</value>
	</property>
	<property>
		<name>dfs.data.dir</name>
		<value>/home/hadoop/fs/data</value>
	</property>
	<property>
		<name>dfs.name.dir</name>
		<value>/home/hadoop/fs/name</value>
	</property>
	<property>
		<name>dfs.back.http.address</name>
		<value>hadoop:50070</value>
	</property>
	<property>
		<name>dfs.permissions</name>
		<value>false</value>
	</property>
</configuration>

mapred-site.xml:

<!-- Put site-specific property overrides in this file. -->
<configuration>
	<property>
		<name>mapred.job.tracker</name>
		<value>hadoop:9001</value>
	</property>
</configuration>

1.首先输入jps，查看hadoop进程，如下：

hadoop@hadoop:~/hadoop-1.0.4/bin$ jps
16018 TaskTracker
15646 DataNode
17250 NameNode
15870 JobTracker
17296 Jps
15790 SecondaryNameNode

2.kill掉namenode的进程，继续查看hadoop进程，如下：

hadoop@hadoop:~/hadoop-1.0.4/bin$ kill 17250
hadoop@hadoop:~/hadoop-1.0.4/bin$ jps
16018 TaskTracker
15646 DataNode
17328 Jps
15870 JobTracker
15790 SecondaryNameNode

3. 删除dfs.name.dir的目录的全部内容；

4. 这里没有按照参考的博客那样把secondarynamenode的fs.checkpoint.dir的内容拷贝到namenode的dfs.name.dir下面，而是直接使用./hadoop namenode -importCheckPoint命令，这个命令应该已经含有了拷贝的命令了，出现下面的logs信息：

hadoop@hadoop:~/hadoop-1.0.4/bin$ ./hadoop namenode -importCheckPoint
13/05/28 15:37:14 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = hadoop/192.168.128.138
STARTUP_MSG:   args = [-importCheckPoint]
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
13/05/28 15:37:14 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/05/28 15:37:14 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/05/28 15:37:14 INFO impl.MetricsSystemImpl: NameNode metrics system started
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/05/28 15:37:14 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source jvm registered.
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source NameNode registered.
13/05/28 15:37:14 INFO util.GSet: VM type       = 32-bit
13/05/28 15:37:14 INFO util.GSet: 2% max memory = 19.33375 MB
13/05/28 15:37:14 INFO util.GSet: capacity      = 2^22 = 4194304 entries
13/05/28 15:37:14 INFO util.GSet: recommended=4194304, actual=4194304
13/05/28 15:37:14 INFO namenode.FSNamesystem: fsOwner=hadoop
13/05/28 15:37:14 INFO namenode.FSNamesystem: supergroup=supergroup
13/05/28 15:37:14 INFO namenode.FSNamesystem: isPermissionEnabled=false
13/05/28 15:37:14 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
13/05/28 15:37:14 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
13/05/28 15:37:15 INFO namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean
13/05/28 15:37:15 INFO namenode.NameNode: Caching file names occuring more than 10 times 
13/05/28 15:37:15 INFO common.Storage: Storage directory /home/hadoop/fs/name is not formatted.
13/05/28 15:37:15 INFO common.Storage: Formatting ...
13/05/28 15:37:15 INFO common.Storage: Number of files = 8
13/05/28 15:37:15 INFO common.Storage: Number of files under construction = 0
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 loaded in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Edits file /home/hadoop/checkpoint/namesecodary/current/edits of size 4 edits # 0 loaded in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO namenode.NameCache: initialized with 0 entries 0 lookups
13/05/28 15:37:15 INFO namenode.FSNamesystem: Finished loading FSImage in 345 msecs
13/05/28 15:37:15 INFO hdfs.StateChange: STATE* Safe mode ON. 
The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
13/05/28 15:37:15 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
13/05/28 15:37:15 INFO impl.MetricsSourceAdapter: MBean for source FSNamesystemMetrics registered.
13/05/28 15:37:15 INFO impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort9000 registered.
13/05/28 15:37:15 INFO impl.MetricsSourceAdapter: MBean for source RpcActivityForPort9000 registered.
13/05/28 15:37:15 INFO ipc.Server: Starting SocketReader
13/05/28 15:37:15 INFO namenode.NameNode: Namenode up at: hadoop/192.168.128.138:9000
13/05/28 15:37:35 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/05/28 15:37:35 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/05/28 15:37:35 INFO http.HttpServer: dfs.webhdfs.enabled = false
13/05/28 15:37:35 INFO http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50070
13/05/28 15:37:35 INFO http.HttpServer: listener.getLocalPort() returned 50070 webServer.getConnectors()[0].getLocalPort() returned 50070
13/05/28 15:37:35 INFO http.HttpServer: Jetty bound to port 50070
13/05/28 15:37:35 INFO mortbay.log: jetty-6.1.26
13/05/28 15:37:35 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:50070
13/05/28 15:37:35 INFO namenode.NameNode: Web-server up at: 0.0.0.0:50070
13/05/28 15:37:35 INFO ipc.Server: IPC Server Responder: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server listener on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 0 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 1 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 2 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 3 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 4 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 5 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 6 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 7 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 8 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 9 on 9000: starting
13/05/28 15:37:35 INFO hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.128.138:50010 storage DS-2106362203-192.168.128.138-50010-1368229536640
13/05/28 15:37:35 INFO net.NetworkTopology: Adding a new node: /default-rack/192.168.128.138:50010
13/05/28 15:37:35 INFO hdfs.StateChange: STATE* Safe mode extension entered. 
The ratio of reported blocks 1.0000 has reached the threshold 0.9990. Safe mode will be turned off automatically in 29 seconds.
13/05/28 15:37:35 INFO hdfs.StateChange: *BLOCK* NameSystem.processReport: from 192.168.128.138:50010, blocks: 1, processing time: 10 msecs
13/05/28 15:37:56 INFO hdfs.StateChange: STATE* Safe mode ON. 
The ratio of reported blocks 1.0000 has reached the threshold 0.9990. Safe mode will be turned off automatically in 9 seconds.
13/05/28 15:38:06 INFO namenode.FSNamesystem: Total number of blocks = 1
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of invalid blocks = 0
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of under-replicated blocks = 1
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of  over-replicated blocks = 0
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Safe mode termination scan for invalid, over- and under-replicated blocks completed in 56 msec
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Leaving safe mode after 51 secs.
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Safe mode is OFF.
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Network topology has 1 racks and 1 datanodes
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* UnderReplicatedBlocks has 1 blocks
13/05/28 15:38:06 INFO namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 1 msec
13/05/28 15:38:06 INFO namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 1 msec processing time, 1 msec clock time, 1 cycles
13/05/28 15:38:06 INFO namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 0 msec
13/05/28 15:38:06 INFO namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 0 msec processing time, 0 msec clock time, 1 cycles
13/05/28 15:38:06 INFO namenode.FSNamesystem: Roll Edit Log from 192.168.128.138
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 
13/05/28 15:38:06 INFO namenode.FSNamesystem: Roll FSImage from 192.168.128.138
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 1 SyncTimes(ms): 5

5.在kill掉namenode后，50070界面就不能访问了，但是使用了4的命令后，这个界面又可以访问了。但是使用CTRL+c后50070页面又不可以访问了，这时使用命令./hadoop-daemon.sh start namenode 后稍等片刻就又可以访问50070界面了。

iteye_12675
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
SecondaryNameNode职责.pptx
05-17
SecondaryNameNode职责
hdfs-secondarynamenode.png
01-28
hdfs-secondarynamenode,nn2节点的作用，以及它的运行原理
Nand Flash Error Management
52yyp的专栏
03-03 320
NVB Each NAND Flash die (LUN) is specified to have a minimum number of valid blocks (NVB) of the total available blocks. This means the die (LUNs) could have blocks that are invalid when shipped from the factory. Bad Block Definition An invalid block i.
HDFS集群的启动(8)——NameNode任务线程之FSNamesystem$SafeModeMonitor
xhh198781的专栏
11-29 3225
HDFS集群启动之后并不能马上进入工作模式为用户提供服务，而是先要进入安全模式状态，因为主节点要确定当前集群中的绝大多数数据块Block处于安全可用的状态以确保为用户提供更可靠的读服务。这个工作主要是交由主节点上的FSNamesystem来完成的。FSNamesystem在初始化的时侯会首先进入安全模式状态并在之后开启一个后台工作线程——SafeModeMonitor来监控当前集群是否可以离开安全
Hadoop 系列YARN：资源调度平台（YARN的命令）
weixin_33814685的博客
11-20 532
yarn命令的用法 [root@node1 ~]# yarn Usage: yarn [--config confdir] [COMMAND | CLASSNAME] CLASSNAME run the class named CLASSNAME or where COMMAND is one of: resou...
Java 内存泄露 Memory Leak（Oracle官方文档）
luoshenFU的专栏
07-12 8103
http://www.javacodegeeks.com/2011/07/java-and-memory-leaks.html http://www.ibm.com/developerworks/library/j-leaks/ http://stackoverflow.com/questions/37335/how-to-deal-with-java-lang-outofmemory
SecondaryNamenode应用摘记
JZ_Kaka的博客
11-04 260
一、环境 Hadoop 0.20.2、JDK 1.6、Linux操作系统 二、背景 上周五的时候，由于操作系统的原因，导致JDK出现莫名的段错误。无论是重启机器还是JDK重装都无济于事。更可悲的是，出问题的机器就是Master。当时心里就凉了半截，因为secondarynamenode配置也是在这个机器上（默认的，没改过）。不过万幸的是这个集群是测试环境，所以问题不大。借这个缘由，我将...
NameNode及SecondaryNameNode分析
07-07
NameNode及SecondaryNameNode分析
快速学习-NameNode和SecondaryNameNode
01-07
第5章 NameNode和SecondaryNameNode（面试开发重点） 5.1 NN和2NN工作机制 思考：NameNode中的元数据是存储在哪里的？ 首先，我们做个假设，如果存储在NameNode节点的磁盘中，因为经常需要进行随机访问，还有响应...
Hadoop中namenode和secondarynamenode工作机制讲解
09-30
今天小编就为大家分享一篇关于Hadoop中namenode和secondarynamenode工作机制讲解，小编觉得内容挺不错的，现在分享给大家，具有很好的参考价值，需要的朋友一起跟随小编来看看吧
Hadoop学习总结之五：Hadoop的运行痕迹
diebiaosang4496的博客
11-23 231
Hadoop 学习总结之一：HDFS简介Hadoop学习总结之二：HDFS读写过程解析Hadoop学习总结之三：Map-Reduce入门Hadoop学习总结之四：Map-Reduce的过程解析在使用hadoop的时候，可能遇到各种各样的问题，然而由于hadoop的运行机制比较复杂，因而出现了问题的时候比较难于发现问题。 本文欲通过某种方式跟踪Hadoop的运行痕迹，方便出现问题的时候...
hadoop3.1.1全分布式配置求助：hadoop结点所有数据为0，不是hosts问题。
lilin201501110113的博客
03-19 2595
求助大神看一下!日志文件，自己看了不懂在这里插入图片描述 hdfs-site.xml core.site.xml在这里插入图片描述在这里插入图片描述 日志文件 2019-03-19 19:57:20,933 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM ...
secondary namenode和hadoop-HA的区别
yhblog的博客
12-04 891
参考：https://blog.csdn.net/jarth/article/details/52839864 https://blog.csdn.net/zhanaolu4821/article/details/82149804
SecondaryNameNode
luyee2010的专栏
03-01 603
SecondaryNameNode有两个作用，一是镜像备份，二是日志与镜像的定期合并。两个过程同时进行，称为checkpoint  总共分五步：  1.SecondaryNameNode通知NameNode准备提交edits文件，此时主节点产生edits.new  2.SecondaryNameNode通过http get方式获取NameNode的fsimage与edits文件（在Secon
Hbase 笔记(10) 集群监控
荒岛码农
11-14 6730
待续
secondarynamenode
最新发布
09-05
Secondary NameNode还负责监控主NameNode的内存使用情况，并在需要时触发主NameNode的内存回收。它并不直接处理客户端请求，而是与主NameNode保持同步并提供辅助功能。总之，Secondary NameNode在Hadoop集群中起到...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值