mr和hbase的整合报错Exception in thread “main“ java.lang.IllegalArgumentException...is not a valid DFS file

最新推荐文章于 2022-05-14 23:45:08 发布

码上行动、

最新推荐文章于 2022-05-14 23:45:08 发布

阅读量398

点赞数

分类专栏： hbase

本文链接：https://blog.csdn.net/LFQ244117370/article/details/118851928

版权

hbase 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

Exception in thread "main" java.lang.IllegalArgumentException: Pathname /D:/maven/apache-maven-3.6.3/repository/org/apache/hbase/thirdparty/hbase-shaded-gson/3.0.0/hbase-shaded-gson-3.0.0.jar from hdfs://qianfeng01:8020/D:/maven/apache-maven-3.6.3/repository/org/apache/hbase/thirdparty/hbase-shaded-gson/3.0.0/hbase-shaded-gson-3.0.0.jar is not a valid DFS filename.
	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:187)
	at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:101)
	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1068)
	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1758)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
	at com.al.hbase1.Hbase0nMr.main(Hbase0nMr.java:97)

TableMapReduceUtil.initTableReducerJob(outputTableName,MyReducer.class,job);
主要是因为你输入的路径是hdfs上的路径，在windows中不识别
修改成
TableMapReduceUtil.initTableReducerJob(outputTableName,MyReducer.class,job,null,null,null,null,false);