HBase Snapshot 迁移: snapshot.ExportSnapshot: Snapshot export failed

最新推荐文章于 2024-04-27 01:42:46 发布

灰二和杉菜

最新推荐文章于 2024-04-27 01:42:46 发布

阅读量2.3k

点赞数

分类专栏： BigData 文章标签： HBase shapshot snapshot.ExportSnapshot: Snapshot e HBase迁移

本文链接：https://blog.csdn.net/qq475781638/article/details/95226553

版权

本文记录了在进行HBase Snapshot迁移时遇到的MapReduce OutOfMemory (OOM)错误，以及如何通过调整MR参数-Dmapreduce.map.memory.mb=4096 -Dmapreduce.map.java.opts=-Xmx3686m来成功解决问题。

摘要由CSDN通过智能技术生成

测试HBase数据迁移时出现MR OOM问题，在这里记录一下解决方法，以防忘记

异常信息

./hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot click-count-snp -copy-to hdfs://172.16.201.42:8020/hbase -mappers 10 -bandwidth 20

2019-07-09 16:01:29,407 INFO  [main] snapshot.ExportSnapshot: Copy Snapshot Manifest
2019-07-09 16:01:29,722 WARN  [main] mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present.  Continuing without it.
2019-07-09 16:01:30,793 INFO  [main] snapshot.ExportSnapshot: Loading Snapshot 'click-count-snp' hfile list
2019-07-09 16:01:31,066 INFO  [main] mapreduce.JobSubmitter: number of splits:1
2019-07-09 16:01:31,173 INFO  [main] mapreduce.JobSubmitter: Submitting tokens for job: job_1562659148784_0003
2019-07-09 16:01:31,588 INFO  [main] impl.YarnClientImpl: Submitted application application_1562659148784_0003
2019-07-09 16:01:31,615 INFO  [main] mapreduce.Job: The url to track the job: http://nn1:8088/proxy/application_1562659148784_0003/
2019-07-09 16:01:31,616 INFO  [main] mapreduce.Job: Running job: job_1562659148784_0003
2019-07-09 16:01:37,758 INFO  [main] mapreduce.Job: Job job_1562659148784_0003 running in uber mode : false
2019-07-09 16:01:37,759 INFO  [main] mapreduce.Job:  map 0% reduce 0%
2