[hdfs@hmaster-1 root]$ hadoop distcp hdfs://hdfs-ha/apps/hbase/data hdfs://hdfs-ha/backup/hbase_bk_20171222
出现的错误摘要:
Caused by: java.io.IOException:Mismatch in length of source:hdfs://hdfs-ha/apps/hbase/data/WALs/core-06/backup/hbase_bk_20171222/.distcp.tmp.attempt_1506497058251_3878_m_000000_1
at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareFileLengths(RetriableFileCopyCommand.java:193)
at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:126)
最大可能的原因是由于文件未关闭而影响了distcp作业执行,验证一下是不是,查看这个文件状态:
[hdfs@hmaster-1 ~]$ hdfs fsck hdfs://hdfs-ha/apps/hbase/data/WALs/core-06,16020,1513713156134-splitting/core-06%2C16020%2C1513713156134.default.1513946044353
Connecting to namenode via http://hmaster-1.ksc.com:50070/fsck?ugi=hdfs&path=%2Fapps%2Fhbase%2Fdata%2FWALs%2Fcore-06%2C16020%2C1513713156134-splitting%2Fcore-06%252C16020%252C1513713156134.default.1513946044353
FSCK started by hdfs (auth:SIMPLE) from /172.31.0.10 for path /apps/hbase/data/WALs/core-06,16020,1513713156134-splitting/core-06%2C16020%2C1513713156134.default.1513946044353 at Fri Dec 22 22:18:06 CST 2017
.Status: HEALTHY
Total size: 16004696 B
Total dirs: 0
Total files: 1
Total symlinks: 0