java只获取hive表列名,Apache Kylin - Re: hive表数据源： 当hive的table中某列名含有中文，build cube 会报错...

Posted by PENG Zhengshuai on

URL: http://apache-kylin.74782.x6.nabble.com/hive-hive-table-build-cube-tp12863p12873.html

Hi,

Can you show the Hive Sql in Kylin.log when cube building?

BR

PENG Zhengshuai

> On Mar 5, 2019, at 3:25 PM, 奥威软件 wrote:

>

> Hi,

> hive表数据源： 当hive的table中某列名含有中文，build cube 会报错

> kylin2.6.0 hadoop3

> 错误如下：

>

>

> NoViableAltException(24@[]) at org.apache.hadoop.hive.ql.parse.HiveParser.type(HiveParser.java:36813) at org.apache.hadoop.hive.ql.parse.HiveParser.colType(HiveParser.java:36595) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeConstraint(HiveParser.java:34322) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrConstraint(HiveParser.java:34075) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrConstraintList(HiveParser.java:29819) at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6662) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:4295) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2494) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1420) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:220) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:74) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:67) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:616) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1826) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1773) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1768) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) FAILED: ParseException line 4:19 cannot recognize input near 'ID' 'int' ',' in column type The command is:  hive -e "USE default; DROP TABLE IF EXISTS kylin_intermediate_ch_cube_5cc15555_a6f3_f9f8_b90a_3e52abe0760b; CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_ch_cube_5cc15555_a6f3_f9f8_b90a_3e52abe0760b ( ICSTOCKBILL_1W_C_门店ID int ,ICSTOCKBILL_1W_C_客户ID int ,ICSTOCKBILL_1W_C_时间 timestamp ,ICSTOCKBILL_1W_C_商品ID int ,GOODS_C_商品ID int ,GOODS_C_品类ID int ,DEPARTMENT_C_门店ID int ,DEPARTMENT_C_区域ID int ,GOODSCLASS_C_品类ID int ,DEPARTMENTCLASS_C_区域ID int ,ICSTOCKBILL_1W_C_数量 int ,ICSTOCKBILL_1W_C_进货价 decimal(20,3) ,ICSTOCKBILL_1W_C_总售价 decimal(20,3) ,ICSTOCKBILL_1W_C_售价 decimal(20,3) ,ICSTOCKBILL_1W_C_总成本 decimal(20,3) ) STORED AS SEQUENCEFILE LOCATION 'hdfs://kylincluster/kylin/kylin_metadata/kylin-6613a735-0452-1bd5-aa22-e63013366c2a/kylin_intermediate_ch_cube_5cc15555_a6f3_f9f8_b90a_3e52abe0760b';

> 是kylin生成的语句缺少了 单反引号 `。

>

>

> 手动给每个表名添加单反引号 `，然后执行上述sql命令，是正常的。

> 正常的sql语句如下：

> CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_ch_cube_5cc15555_a6f3_f9f8_b90a_3e52abe0760b

> (

> `ICSTOCKBILL_1W_C_mdID` int

> ,`ICSTOCKBILL_1W_C_khID` int

> ,`ICSTOCKBILL_1W_C_sj` timestamp

> ,`ICSTOCKBILL_1W_C_spID` int

> ,`GOODS_C_spID` int

> ,`GOODS_C_plID` int

> ,`DEPARTMENT_C_mdID` int

> ,`DEPARTMENT_C_qyuID` int

> ,`GOODSCLASS_C_plID` int

> ,`DEPARTMENTCLASS_C_quyID` int

> ,`ICSTOCKBILL_1W_C_shul` int

> ,`ICSTOCKBILL_1W_C_jhj` decimal(20,3)

> ,`ICSTOCKBILL_1W_C_zsj` decimal(20,3)

> ,`ICSTOCKBILL_1W_C_sj` decimal(20,3)

> ,`ICSTOCKBILL_1W_C_zcb` decimal(20,3)

> )

> STORED AS SEQUENCEFILE;

>

>

>

> 请帮忙看下怎么解决，谢谢！

>

>

> Best regards

> --

> Regards!

> Aron Tao