大数据错误

我是老孙

已于 2023-09-04 16:30:35 修改

阅读量667

点赞数

文章标签：大数据 linux 数据库

于 2023-09-04 16:30:02 首次发布

本文链接：https://blog.csdn.net/sunjiyun26/article/details/132662739

版权

question1 : Could not locate Hadoop executable: D:\hadoop-3.3.1\bin\winutils.exe -

【已解决】Could not locate executable E:\Hadoop\bin\winutils.exe in the Hadoop binaries._could not locate executable e:\hadoop-3.3.1\bin\wi_君问归期魏有期的博客-CSDN博客

question2

java.lang.Exception: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1332067403-192.168.159.129-1692011466914:blk_1073741843_1019 file=/test/0417.sql
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) ~[hadoop-mapreduce-client-common-3.3.1.jar:na]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) ~[hadoop-mapreduce-client-common-3.3.1.jar:na]
Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1332067403-192.168.159.129-1692011466914:blk_1073741843_1019 file=/test/0417.sql

改成域名设置

org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
conf.set("dfs.replication", "1");
conf.set("dfs.client.use.datanode.hostname", "true");
conf.set("mapred.job.tracker", "hdfs://hadoop-master:8020/");
conf.set("fs.defaultFS", "hdfs://hadoop-master:8020/");

标红的地方修改成域名，如果原来的时候写的是IP

并且在 C:\Windows\System32\drivers\etc\hosts 加入hosts 解析，例如

192.168.159.129 hadoop-master
192.168.159.133 hadoop-slave1
192.168.159.134 hadoop-slave2