在使用sqoop用hcatalog从mysql导数据到hive时,我又碰到一个问题:
NoSuchObjectException(message:default.simple table not found)
具体的trace是
NoSuchObjectException(message:default.bam_webapp_statistics table not found)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1227)at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1222)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:360)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1222)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:734)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:901)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:843)
...
经过一阵google,发现是sqoop没有连接到正确的hive配置的锅。
于是在sqoop/conf文件夹下执行 cp sqoop-env-template.sh sqoop-env.sh。
然后,修改sqoop-env.sh,将
export HADOOP_COMMON_HOME=/usr/local/hadoop/hadoop
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/usr/local/hadoop/hadoop
#set the path to where bin/hbase is available
#export HBASE_HOME=
#Set the path to where bin/hive is available
export HIVE_HOME=/usr/local/hadoop/hive
#Set the path for where zookeper config dir is
#export ZOOCFGDIR=
把这些路径都给配置上,问题就解决了。