mysql 创建表带编码_hive创建表带中文注释报错解决方法

最新推荐文章于 2023-05-10 10:58:25 发布
最新推荐文章于 2023-05-10 10:58:25 发布
阅读量286 收藏
点赞数
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_39995764/article/details/113458324
版权

Hive MySQL 编码问题 中文注释 解决方法
关键词由CSDN通过智能技术生成

hive创建带有中文注释的表报错；

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Transaction failed to commit

at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)

at org.datanucleus.api.jdo.JDOTransaction.rollback(JDOTransaction.java:186)

at org.apache.hadoop.hive.metastore.ObjectStore.rollbackTransaction(ObjectStore.java:444)

at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:746)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)

at com.sun.proxy.$Proxy0.createTable(Unknown Source)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1270)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1303)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:106)

at com.sun.proxy.$Proxy12.create_table_with_environment_context(Unknown Source)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:8016)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:8000)

at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)

at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

NestedThrowablesStackTrace:

Exception thrown flushing changes to datastore

org.datanucleus.exceptions.NucleusDataStoreException: Exception thrown flushing changes to datastore

at org.datanucleus.store.rdbms.SQLController$1.transactionFlushed(SQLController.java:708)

at org.datanucleus.store.connection.AbstractManagedConnection.transactionFlushed(AbstractManagedConnection.java:86)

at org.datanucleus.store.connection.ConnectionManagerImpl$2.transactionFlushed(ConnectionManagerImpl.java:454)

at org.datanucleus.TransactionImpl.flush(TransactionImpl.java:203)

at org.datanucleus.TransactionImpl.rollback(TransactionImpl.java:420)

at org.datanucleus.api.jdo.JDOTransaction.rollback(JDOTransaction.java:182)

at org.apache.hadoop.hive.metastore.ObjectStore.rollbackTransaction(ObjectStore.java:444)

at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:746)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)

at com.sun.proxy.$Proxy0.createTable(Unknown Source)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1270)

at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1303)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:106)

at com.sun.proxy.$Proxy12.create_table_with_environment_context(Unknown Source)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:8016)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:8000)

at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)

at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after statement closed.

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)

at com.mysql.jdbc.Util.getInstance(Util.java:386)

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)

at com.mysql.jdbc.StatementImpl.checkClosed(StatementImpl.java:447)

at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1411)

at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:424)

at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeBatch(ParamLoggingPreparedStatement.java:372)

at org.datanucleus.store.rdbms.SQLController.processConnectionStatement(SQLController.java:628)

at org.datanucleus.store.rdbms.SQLController.processStatementsForConnection(SQLController.java:596)

at org.datanucleus.store.rdbms.SQLController$1.transactionFlushed(SQLController.java:683)

... 30 more

Nested Throwables StackTrace:

character_set_database   需要设置成“latin1”

修改数据库编码：

mysql> set character_set_database =latin1;

Query OK, 0 rows affected (0.00 sec)

mysql> show variables like '%char%';

+--------------------------+----------------------------+

| Variable_name | Value |

+--------------------------+----------------------------+

| character_set_client | utf8 |

| character_set_connection | utf8 |

| character_set_database | latin1 |

| character_set_filesystem | binary |

| character_set_results | utf8 |

| character_set_server | utf8 |

| character_set_system | utf8 |

| character_sets_dir | /usr/share/mysql/charsets/ |

+--------------------------+----------------------------+

8 rows in set (0.00 sec)

为了支持Hive建表时插入中文注释 需要在MySQL中做如下设置：

use hive;

//修改字段注释字符集

alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;

//修改表注释字符集

alter table TABLE_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;

//修改分区注释字符集

alter table PARTITION_KEYS modify column PKEY_COMMENT varchar(4000) character set utf8;

再次执行建表语句，成功！！！

hive> create table page_view

> (

> page_id bigint comment '页面ID',

> page_name string comment '页面名称',

> page_url string comment '页面URL'

> )

> comment '页面视图'

> partitioned by (ds string comment '当前时间，用于分区字段')

> row format delimited

> stored as rcfile

> location '/user/hive/test';

OK

Time taken: 0.493 seconds

hive> show tables;

OK

page_view

Time taken: 0.057 seconds, Fetched: 1 row(s)

hive> desc page_view;

OK

page_id bigint 页面ID

page_name string 页面名称

page_url string 页面URL

ds string 当前时间，用于分区字段

# Partition Information

# col_name data_type comment

ds string 当前时间，用于分区字段

Time taken: 0.19 seconds, Fetched: 9 row(s)

参考：http://blog.csdn.net/levy_cui/article/details/51646145

weixin_39995764
关注
hive初始化mysql失败_hive 未初始化元数据库报错
weixin_29113833的博客
01-27 2630
启动hive-metastore和hive-server2 用beeline连接hive报错[root@node04 hive]# beelineBeeline version 0.13.1-cdh5.3.0 by Apache Hivebeeline> !connect jdbc:hive2://172.16.145.114:10000 hivescan complete in 3msCo...
hive与mysql的语法区别_HIVE基本语法(创建删除修改表和表结果)以及分区详细讲解...
weixin_39622980的博客
01-27 1266
HIVE小结HIVE基本语法HIVE和Mysql十分类似建表规则CREATE [EXTERNAL] TABLE [IF NOT EXISTS] table_name[(col_name data_type [COMMENT col_comment], ...)][COMMENT table_comment][PARTITIONED BY (col_name data_type [COMMENT c...
mysql 创建表带编码_字符串变表带有表头
weixin_32616931的博客
01-28 162
Sqlserver 字符串变为一个表 可以根据需求自行更改，灵活运用 无 declare @LSTR_InsertHead varchar(500)--列名declare @LSTR_InsertFields varchar(max)--数据字符串declare @splitrol varchar(2)--行分割符declare @splitcol varchar(2)--列分隔符set @spl...
mysql 创建表带编码_Python3-sqlalchemy-orm 创建关联表带外键并插入数据
weixin_39908070的博客
02-01 140
/Users/drizzle/PycharmProjects/2018-12-03/venv/bin/python /Users/drizzle/PycharmProjects/2018-12-03/orm_fk.py2019-01-08 18:08:40,496 INFO sqlalchemy.engine.base.Engine SHOW VARIABLES LIKE 'sql_mode'20...
Hive启动报错Caused by MetaException message Version information not found in metastore
热门推荐
NeverGiveup54的专栏
01-30 1万+
1问题错误： Caused by: MetaException(message:Versioninformation not found in metastore. ) 或 Caused by: javax.jdo.JDODataStoreException:Required table missing : "`VERSION`" in Catalog "" Schema"". DataNu
spark插入动态分区代码报错
cclovezbf的博客
11-07 2420
config("hive.metastore.dml.events","false") 就好了。windows确实好了。.config("hive.exec.dynamic.partition.mode", "nonstrict")//动态分区。.config("hive.metastore.dml.events","false")//动态分区。但是我在服务器跑代码反复测试 都是可以重复插入分区的。window实在是难搞。dataset第一次插入分区没有问题，但是第二次重新跑的时候。
hiveserver2启动：javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS""
小太阳~
07-16 1万+
原文链接：http://www.cnblogs.com/zwgblog/p/6063993.html 启动hive-metastore和hive-server2 用beeline连接hive报错 1 2 3 4 5 6 7 8 [root@node04 hive]# beeline Bee
Mysql元数据如何生成Hive建表语句注释脚本详解
01-19
最近在将数据从Mysql 等其他关系型数据库 抽取到Hive 表中时，需要同步mysql表中的注释，以下脚本可以生成hive表字段注释修改语句。 注：其他关系型数据库如：oracle 可以通过相同的思路，读取元数据，修改脚本语法...
Hive安装讲义(linux_hive linux_mysql)
09-14
Hive安装讲义(linux_hive linux_mysql) Hive安装讲义中涵盖了Hive的安装过程，包括Hadoop和MySQL的安装。下面是从给定的文件中生成的相关知识点： 一、Hadoop 安装 * Hadoop 安装是Hive 运行环境的前提条件 * ...
hive 创建访问用户_hive创建角色并赋权
weixin_33923901的博客
02-23 4167
1 角色管理命令1.1 创建角色创建一个新角色，需要 admin 用户执行CREATE ROLE role_name;1.2 删除角色删除一个角色，需要 admin 用户执行DROP ROLE role_name;1.3 显示当前角色显示用户当前角色列表SHOW CURRENT ROLES;1.4 设定角色如果指定了role_name，则该角色将成为当前角色中的唯一角色将Role_Name设置为A...
spark-sql 报错：Exception thrown flushing changes to datastore
赵英超的博客
05-10 877
spark-sql hive
【mq918一起学hive】hive metaException报错分析
bulbcat的博客
11-23 1148
hive 异常处理 MetaException
sparkSession访问hive，PARTITIONS表不可用
weixin_42737797的博客
08-15 393
为分布式搭建hive on spark; 记录耗时最久的一个问题：太气人了 2019-08-15 18:53:03 ERROR Datastore:115 - Error thrown executing ALTER TABLE `PARTITIONS` ADD COLUMN `TBL_ID` BIGINT NULL : Table 'hive.PARTITIONS' doesn't e...
hive创建表带中文注释报错解决方法
weixin_33947521的博客
11-15 563
hive创建带有中文注释的表报错； FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Transaction failed to commit at...
Hive客户端多并发问题解决方法记录
weixin_30954607的博客
08-08 515
问题出现场景： 如用户描述：（ 测试脚本如下: echo " set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict; set hive.optimize.sort.dynamic.partition=false; set hive.exec.m...
hive报错MetaException(message:An exception was thrown while adding/validating class
小黑的博客
04-11 5083
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:An exception was thrown while adding/validating class(es) : Column length too big for column '...
hive问题及解决1
dy_252的专栏
05-03 1160
1、hive使用mysql存储元数据，报错：hive> show tables; FAILED: Error in metadata: javax.jdo.JDODataStoreException: Error(s) were found while auto-creating/validating the datastore for classes. The errors are print
hive中创建数据库,添加中文注释乱码如何解决
最新发布
06-10
在Hive中创建数据库时，可以使用以下命令来添加中文注释： ``` CREATE DATABASE my_database COMMENT '中文注释' ``` 如果中文注释出现乱码，可能是因为Hive默认使用的字符集不支持中文。可以通过在Hive配置文件...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值