本来想用kettle,把hive数仓的DM层结果数据导出到clickhouse的表中。结果hive、clickhouse,kettle都可以连上。但是依然报错,搞了半天还没搞定,头大。
先上clickhouse表输出控件的输入字段映射的报错截图
一、hive和clickhouse数据库,kettle本身都可以连上
1.kettle连接hive
2.kettle连接clickhouse
二、kettle转换任务设置
1.表输入插件(hive表输入)
hive表输入没问题
2.字段选择插件
可以选择字段,正常
3.表输出插件(输出到clickhouse)
(1)表输出主选项正常
(2)获取字段正常,但是输入字段映射报错
(3)输入字段报错
Couldn't get field info from [hurys_dc.dm_evaluation_15min]
ClickHouse exception, code: 1002, host: 192.168.0.22, port: 8123; Code: 62. DB::Exception: Syntax error: failed at position 1 ('hurys_dc'): hurys_dc.dm_evaluation_15min. Expected one of: ALTER query, Query with output, ALTER PROFILE, RENAME DATABASE, SHOW PRIVILEGES query, TRUNCATE, KILL, KILL QUERY query, SELECT query, possibly with UNION, list of union elements, ALTER ROLE, SELECT subquery, DESCRIBE query, SELECT query, subquery, possibly with UNION, SHOW GRANTS, SHOW CREATE, CREATE SETTINGS PROFILE or ALTER SETTINGS PROFILE query, WATCH, SHOW PROCESSLIST query, ALTER POLICY, ALTER USER, CREATE VIEW query, CHECK TABLE, SET ROLE, SELECT query, SELECT, REVOKE, CREATE USER, CREATE DICTIONARY, CREATE PROFILE, SET ROLE DEFAULT, ALTER SETTINGS PROFILE, EXPLAIN, SYSTEM, ALTER LIVE VIEW, RENAME TABLE, DROP query, SHOW ACCESS, OPTIMIZE query, USE, DROP access entity query, RENAME DICTIONARY, DETACH, SET, SHOW, DESC, OPTIMIZE TABLE, CREATE ROW POLICY, SET DEFAULT ROLE, EXCHANGE DICTIONARIES, CREATE POLICY, ALTER ROW POLICY, INSERT INTO, INSERT query, SHOW [TEMPORARY] TABLES|DATABASES|CLUSTERS|CLUSTER 'name' [[NOT] [I]LIKE 'str'] [LIMIT expr], GRANT, RENAME query, SHOW GRANTS query, SHOW PRIVILEGES, EXISTS, DROP, SYSTEM query, CREATE LIVE VIEW query, CREATE ROW POLICY or ALTER ROW POLICY query, CREATE QUOTA or ALTER QUOTA query, SHOW PROCESSLIST, ALTER QUOTA, CREATE QUOTA, CREATE DATABASE query, SET query, Query, CREATE, WITH, CREATE ROLE or ALTER ROLE query, EXTERNAL DDL FROM, EXCHANGE TABLES, EXISTS or SHOW CREATE query, WATCH query, REPLACE, CREATE ROLE, CREATE SETTINGS PROFILE, SET ROLE or SET DEFAULT ROLE query, CREATE USER or ALTER USER query, EXTERNAL DDL query, SHOW ACCESS query, SHOW CREATE QUOTA query, USE query, ATTACH, DESCRIBE, ALTER TABLE, ShowAccessEntitiesQuery, GRANT or REVOKE query, CREATE TABLE or ATTACH TABLE query. (SYNTAX_ERROR) (version 21.9.5.16 (official build))
查看错误细节
找了半天,到处百度、搜博客,目前都没解决。希望能有大神帮忙,先在此拜谢!!! ——2023/05/06
后续 2023/05/29——已解决
已经找到了原因,是kettle的版本问题。kettle8.2就会这样,kettle9.3就自动解决了,可以字段映射。不过,即使kettle8.2,它也不影响任务运行,kettle8.2也可以用。