sqoop将mysql数据导入hive时因缺少jar包报的一些错误以及解决方法

最新推荐文章于 2022-11-07 16:11:00 发布
最新推荐文章于 2022-11-07 16:11:00 发布
阅读量1.7k 收藏 6
点赞数 1
文章标签： sqoop jar hive
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u013593554/article/details/121027707
版权

错误一：缺少log4j相关jar包导致的错误

21/10/28 17:32:27 ERROR tool.ImportTool: Import failed: java.io.IOException: Exception thrown in Hive
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:358)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331)
	... 9 more
Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerContextFactory
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:657)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
	... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.spi.LoggerContextFactory
	at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 16 more

错误二：缺少datanucleus jar包导致的错误

21/10/28 17:46:21 WARN metadata.Hive: Failed to register all functions.
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3367)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3406)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3386)
	at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3640)
	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:236)
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:221)
	at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:366)
	at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:310)
	at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:290)
	at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:266)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:558)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:531)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1652)
	... 31 more
Caused by: java.lang.NoClassDefFoundError: org/datanucleus/NucleusContext
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:340)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.getClass(MetaStoreUtils.java:1620)
	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:64)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:599)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:564)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:626)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:416)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6490)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:238)
	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
	... 36 more
Caused by: java.lang.ClassNotFoundException: org.datanucleus.NucleusContext
	at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 49 more
21/10/28 17:46:21 ERROR tool.ImportTool: Import failed: java.io.IOException: Exception thrown in Hive
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:358)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331)
	... 9 more
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:591)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:531)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
	... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:226)
	at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:366)
	at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:310)
	at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:290)
	at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:266)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:558)
	... 17 more
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3367)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3406)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3386)
	at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3640)
	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:236)
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:221)
	... 22 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1652)
	... 31 more
Caused by: java.lang.NoClassDefFoundError: org/datanucleus/NucleusContext
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:340)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.getClass(MetaStoreUtils.java:1620)
	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:64)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:599)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:564)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:626)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:416)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6490)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:238)
	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
	... 36 more
Caused by: java.lang.ClassNotFoundException: org.datanucleus.NucleusContext
	at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 49 more

错误三：还因为缺少hive的jar包导致报错，xshell中日志被顶没了，错误代码就不放了

针对错误一、二、三的解决方法：
将/usr/hive/apache-hive-2.1.1-bin/lib目录下的包全部复制到/usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/lib当中，已经存在的跳过即可，如👇图所示：
在这里插入图片描述

错误四：jdk中/jre/lib/security/java.policy文件配置问题

21/10/28 17:49:45 ERROR bonecp.BoneCP: Unable to start/stop JMX
java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:457)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at com.jolbox.bonecp.BoneCP.registerUnregisterJMX(BoneCP.java:528)
	at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:500)

解决方法：在该文件末尾加入以下代码：

permission javax.management.MBeanTrustPermission "register";

如图所示👇
在这里插入图片描述

高哥的代码没Bug
关注
  • 1
    点赞
  • 踩
  • 6
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 1
    评论
sqoop批量脚本导入报错ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for C
lsp的博客
01-16 8571
问题原因：数据库的IP/用户名/密码/数据库名称 错误导致连接不上 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter         at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.ja...
import java.io后报错_sqoop mysql--->hive 报错 （ERROR tool.ImportTool: Import failed: java.io.IOExceptio...
weixin_33089345的博客
02-16 397
18/08/22 13:30:53 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConfat org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveC...
ERROR tool.ImportTool: Import failed: java.io.IOException: Hive does not support the SQL type for co
小海的专栏
07-21 5566
使用Sqoop导出oracle数据到hive的时候遇到一个报错，意思是hive不知支持oracle的这个字段类型。 ERROR tool.ImportTool: Import failed: java.io.IOException: Hive does not support the SQL type for column CONT_DOC_DESC ERROR tool.Impo
sqoop--＞hive导入数据遇到的问题；拒绝连接和ERROR tool.ImportTool: Import failed: java.io.IOException （status44）
qq_45112156的博客
07-22 2166
问题一；拒绝链接 20/07/22 10:58:33 INFO mapreduce.Job: map 0% reduce 0% 20/07/22 10:58:36 INFO mapreduce.Job: Task Id : attempt_1595382831037_0006_m_000000_0, Status : FAILED Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions
sqoop搭建与使用,sqoop条件导入、增量导入，mysql至hdfs、hive、hbase，hdfs至mysql，生成jar包到本地
Mogeko1的博客
11-07 1924
sqoop搭建与使用,sqoop条件、增量导入，mysql至hdfs、hive、hbase，hdfs至mysql与怎么生产jar包到本地
工作中的一些错误解决记录
u011496538的博客
01-21 1万+
1) 2018-10-19 CDH OOZIE UI页面打不开： 解决： 把下载好的ext-2.2.zip拷贝到Oozie主机的/var/lib/oozie目录下 运行命令unzip ext-2.2.zip解压（没有包就下载） 2)运行spark2报错： ava.lang.NoClassDefFoundError: scala/runtime/LambdaDe...
hive2.1.1_mysql_sqoop.zip
10-25
总结来说，"hive2.1.1_mysql_sqoop.zip"压缩包为数据仓库搭建提供了一整套解决方案，包括Hive 2.1.1用于数据处理和分析，Sqoop 1.4.7用于数据在MySQL和Hadoop之间的传输，以及MySQL 5.1.47的Java连接器确保两者之间...
Sqoop 导入数据异常处理
01-07
写在前面 我在直接下载https://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.7.0.tar.gz 解压使用会出现几个bug，之前也是出现了，然后也没有来...这是因为sqoop缺少java-json.jar包 下载java-json.jar包： http
sqoop-jar.zip
06-02
在这个名为“sqoop-jar.zip”的压缩包中，包含了Sqoop 1.4.7和1.4.6两个版本的jar文件，以及与Hive相关的jar包，这些文件对于在Hadoop环境中与MySQL等数据库进行交互非常有用。 1. **Sqoop基本概念** - **Sqoop**:...
Sqoop空指针异常需要的jar包.rar
08-21
Sqoop从MySQL导数据到HDFS上，如果报错空指针异常，需要的jar包。正常情况只要安装java-json.jar就可以。如果不行就都安装上。如果你用的是阿里云的EMR，安装完之后还报错的话，需求确认你的jar包中是否含有hive开头...
mysql-connector-java-8.0.13.jar
12-22
本文将深入探讨MySQL Connector/J 8.0.13的核心功能、安装与配置、使用方法以及与其他大数据工具如Hive和Sqoop的集成。 一、MySQL Connector/J 8.0.13核心功能 MySQL Connector/J是一个实现了Java Database ...
解决Hive启动后报错FAILED:HiveException java.lang.RuntimeException
weixin_46964107的博客
06-23 1万+
解决Hive启动后报错FAILED:HiveException java.lang.RuntimeException和MetaException(message:Required table missing : “DBS” in Catalog “” Schema “”. DataNucleus requires this table to perform its persistence operations.
sqoop数据导入hive 遇到的问题
asd1456732891的博客
07-19 1564
1.ERROR exec.DDLTask: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader; 造成的原因：sqoop与hive使用的jackson包的版本...
报错:java.lang.RuntimeException: Unable to instantiate activity ComponentInfo{com.example.d
houyingshang的博客
06-07 4891
刚重新安装了Studio  运行了以前的项目,明明能运行的啊 ! 怎么回突然间报错呢?稍加一看,唉 真是.....太尴尬了报错如下:java.lang.RuntimeException: Unable to instantiate activity ComponentInfo{com.example.demo2/com.example.demo2.v.activity.BaseActivity$M...
sqoop mysql jar包_sqoop部署及使用
weixin_39737001的博客
01-30 575
1、下载sqoopsqoop有两个方向的版本，sqoop 1版本和sqoop 2版本，这里使用sqoop 1。sqoop 1下载链接http://www.apache.org/dyn/closer.lua/sqoop/1.4.72、解压cd /data/tar -zxvfsqoop-1.4.7.bin__hadoop-2.6.0.tar.gz3、修改配置文件，主要是配置hadoop目录和hiv...
[Hive]那些年我们踩过的Hive坑
SmartSi
06-12 3万+
（1）问题一  首先，采用这个命令： hive -hiveconf hive.root.logger=DEBUG,console 可以查看详细信息然后分析可知，缺少mysql的jar包，下载mysql-connector-java-5.1.32.tar.gz，并进行一下操作。 下一步操作，添加jar包： （2）问题二
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.me
热门推荐
W
07-30 3万+
启动hive之后执行查询语句报错如下： 分析得知：hive的数据库MySQL在安装的时候没有初始化 解决办法：初始化数据库 schematool -dbType mysql -initSchema 出现下面这个界面则表示初始化成功 验证： ...
Sqoop异常解决ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No
烂笔头的专栏
11-03 1万+
最近在做mysql数据导入hive的工作，先前导入的时候没有问题，这两天导另外一张表总是提示 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter 问题，网上有说是mysql-connector-j
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang. RuntimeExcep
Chrogean的博客
03-18 2644
升级hive之后建表使用时会有如下报错： 这是由于升级之后元数据库需要重新初始化，而元数据苦是存在mysql中的。所以需要： 第一步：删除旧的元数据库 1.删除metastore_db文件(这一项是用derby做元数据库才会生成的，一般嫌它容量小不会使用，所以没有这个文件。) 2.删除数据库中对应的database 登录mysql mysql -u root -p 删除元数据库，可以从配置文件...
sqoop查询mysql数据导入数据到hive的脚本
最新发布
05-05
以下是一个简单的 Sqoop 查询 MySQL 数据并将其导入到 Hive 的脚本： ``` sqoop import \ --connect jdbc:mysql://<MySQL主机名或IP地址>/<数据库名称> \ --username <MySQL用户名> \ --password <MySQL密码> \ --table <MySQL表名> \ --hive-import \ --hive-table <Hive表名> \ --create-hive-table \ --hive-overwrite \ --null-string '\\N' \ --null-non-string '\\N' \ --fields-terminated-by ',' ``` 其中： - `<MySQL主机名或IP地址>`：MySQL 数据库所在主机的 IP 地址或主机名 - `<数据库名称>`：要查询的 MySQL 数据库的名称 - `<MySQL用户名>`：连接 MySQL 数据库的用户名 - `<MySQL密码>`：连接 MySQL 数据库的密码 - `<MySQL表名>`：要导入到 Hive 中的 MySQL 表的名称 - `<Hive表名>`：导入到 Hive 中的表的名称 - `--create-hive-table`：如果 Hive 中不存在指定的表，则创建该表 - `--hive-overwrite`：如果导入的表已经存在于 Hive 中，则覆盖它 - `--null-string '\\N'` 和 `--null-non-string '\\N'`：将 MySQL 中的空值（null）转换为 Hive 中的字符串 '\\N' - `--fields-terminated-by ','`：指定字段分隔符为逗号 请注意，您需要在执行此脚本之前，确保已经在 Hive 中创建了指定的表，并且 Hive 和 MySQL 数据库都可以正常连接。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

高哥的代码没Bug

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值