Error while compiling statement: FAILED: SemanticException Line 1:17 Invalid path

dogplays

于 2023-11-17 17:24:07 发布

阅读量849

点赞数

分类专栏： sqoop 文章标签： sqoop hive

本文链接：https://blog.csdn.net/qq_44540985/article/details/134467270

版权

sqoop 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

再使用sqoop导入mysql数据到hive时，出现以下ERROR

23/11/17 17:08:04 INFO hive.HiveImport: 0: jdbc:hive2://ambari-hadoop1:2181,ambari-ha> LOAD DATA INPATH 'hdfs://ambari-hadoop1:8020/user/hdfs/Test1' OVERWRITE INTO TABLE `test`.`Test1`;
23/11/17 17:08:04 INFO hive.HiveImport: Error: Error while compiling statement: FAILED: SemanticException Line 1:17 Invalid path ''hdfs://ambari-hadoop1:8020/user/hdfs/Test1'' (state=42000,code=40000)
23/11/17 17:08:04 INFO hive.HiveImport: Closing: 0: jdbc:hive2://ambari-hadoop1:2181,ambari-hadoop3:2181,ambari-hadoop2:2181/default;password=123456;serviceDiscoveryMode=zooKeeper;user=hive;zooKeeperNamespace=hiveserver2
23/11/17 17:08:05 ERROR tool.ImportTool: Import failed: java.io.IOException: Hive exited with status 2
	at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:253)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:206)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:273)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:564)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:655)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:151)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:187)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:241)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:250)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:259)

这是sqoop脚本：

sqoop import \
  -Dorg.apache.sqoop.splitter.allow_text_splitter=true \
  --connect jdbc:mysql://ambari-hadoop1:3306/Test \
  --username root \
  --password 123456 \
  --table Test1 \
  --hive-import \
  --hive-database test \
  --hive-table Test1 \
  --hive-overwrite \
  --delete-target-dir

查看hdfs://ambari-hadoop1:8020/user/hdfs/Test1路径是否存在以及是否能ping通ambari-hadoop1：

[hdfs@ambari-hadoop3 log]$ hdfs dfs -ls /user/hdfs/Test1
Found 7 items
-rw-r--r--   3 hdfs hdfs          0 2023-11-17 17:07 /user/hdfs/Test1/_SUCCESS
-rw-r--r--   3 hdfs hdfs          0 2023-11-17 17:07 /user/hdfs/Test1/part-m-00000
-rw-r--r--   3 hdfs hdfs    1564410 2023-11-17 17:07 /user/hdfs/Test1/part-m-00001
-rw-r--r--   3 hdfs hdfs     939692 2023-11-17 17:07 /user/hdfs/Test1/part-m-00002
-rw-r--r--   3 hdfs hdfs          0 2023-11-17 17:07 /user/hdfs/Test1/part-m-00003
-rw-r--r--   3 hdfs hdfs          0 2023-11-17 17:07 /user/hdfs/Test1/part-m-00004
-rw-r--r--   3 hdfs hdfs         52 2023-11-17 17:07 /user/hdfs/Test1/part-m-00005
[hdfs@ambari-hadoop3 log]$ ping ambari-hadoop1
PING ambari-hadoop1 (192.168.10.101) 56(84) bytes of data.
64 bytes from ambari-hadoop1 (192.168.10.101): icmp_seq=1 ttl=64 time=0.546 ms
64 bytes from ambari-hadoop1 (192.168.10.101): icmp_seq=2 ttl=64 time=0.604 ms
64 bytes from ambari-hadoop1 (192.168.10.101): icmp_seq=3 ttl=64 time=1.11 ms
^C
--- ambari-hadoop1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.546/0.753/1.111/0.255 ms
[hdfs@ambari-hadoop3 log]$

以上两者没有问题。但是发现目录文件权限只能hdfs读写，其他用户只能读，那么猜测是否为权限问题

通过ranger权限管控，赋予hive用户具有所有hdfs文件的读写执行权限：