hue集成遇到的问题

最新推荐文章于 2022-04-18 11:55:45 发布

101之歌

最新推荐文章于 2022-04-18 11:55:45 发布

阅读量1.2k

点赞数

分类专栏：大数据

本文链接：https://blog.csdn.net/qxf1374268/article/details/79321963

版权

大数据专栏收录该内容

27 篇文章 0 订阅

订阅专栏

sqoop 报错：Could not get connectors

拷贝sqoop.properties 文件至 /etc/sqoop2/conf 目录

sqoop 报错：Could not start job.

报错：
Could not start job.
exception ERROR {“message”:null,”error-code-class”:”org.apache.sqoop.error.code.GenericJdbcConnectorError”,”stack-trace”:[{“file”:”GenericJdbcFromInitializer.java”,”line”:147,”class”:”org.apache.sqoop.connector.jdbc.GenericJdbcFromInitializer”
解决办法：

cd /opt/cloudera/parcels/CDH/lib/hadoop/client
sudo ln -s ../../hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar .
sudo ln -s ../../hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar .

log4j

Thu Apr 20 10:29:05 CST 2017
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
CONF_DIR=/run/cloudera-scm-agent/process/238-sqoop-SQOOP_SERVER
CMF_CONF_DIR=/etc/cloudera-scm-agent
log4j:ERROR A “org.apache.log4j.RollingFileAppender” object is not assignable to a “org.apache.log4j.Appender” variable.
log4j:ERROR The class “org.apache.log4j.Appender” was loaded by
log4j:ERROR [org.apache.catalina.loader.StandardClassLoader@52437b9a] whereas object of type
log4j:ERROR “org.apache.log4j.RollingFileAppender” was loaded by [WebappClassLoader
context: /sqoop
delegate: false
repositories:
/WEB-INF/classes/
———-> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@52437b9a
].
log4j:ERROR Could not instantiate appender named “RFA”.
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.NativeCodeLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See [url]http://logging.apache.org/log4j/1.2/faq.html#noconfig[/url] for more info.

解决
这个问题，是由于log的jar包冲突导致。可以直接删除log的jar包。
cd /opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/sqoop2/webapps/sqoop/WEB-INF/lib
mv log4j-1.2.16.jar log4j-1.2.16.jar.bak

刚开始我修改的/var/lib/sqoop2/tomcat-deployment/webapps/sqoop 目录下的pom文件和jar包，发现每次重启sqoop2，这个目录下的文件都会被覆盖，后来，通过 find / -name sqoop.war命令查找到/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/sqoop2/ 这个目录下也存在，测试修改了下这个目录下的文件，生效了。OK，搞定。

sqoop新建数据源报错：Could not create link.

{“message”:null,”cause”:{“message”:”The statement was aborted because it would have caused a duplicate key value in a unique or primary key constraint or unique index identified by ‘FK_SQ_LNK_NAME_UNIQUE’ defined on ‘SQ_LINK’.”,”cause”:{“message”:”The statement was aborted because it would have caused a duplicate key value in a unique or primary key constraint or unique index identified by ‘FK_SQ_LNK_NAME_UNIQUE’ defined on ‘SQ_LINK’.”

*解决

sudo -u postgres psql

postgres=# CREATE ROLE sqoop LOGIN ENCRYPTED PASSWORD ‘sqoop’ NOSUPERUSER INHERIT CREATEDB NOCREATEROLE;

postgres=# CREATE DATABASE “sqoop” WITH OWNER = sqoop
ENCODING = ‘UTF-8’
TABLESPACE = pg_default
LC_COLLATE = ‘en_US.UTF-8’
LC_CTYPE = ‘en_US.UTF-8’
CONNECTION LIMIT = -1;