在往 2.0版本做数据迁移时碰到check-sum错误
Caused by: java.io.IOException: Check-sum mismatch between
at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareCheckSums(RetriableFileCopyCommand.java:144)
at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:108)at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:83)
at org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
... 11 more
cdh官网上提供二种解决方案
1. 修改hdfs-site.xml
dfs.checksum.type = CRC32
2.加 -skipcrccheck参数