hdfs mysql sqoop 失败_从hdfs到MySQL的sqoop导出失败,需要帮助以转义字符

bd96500e110b49cbb3cd949968f18be7.png

I have a file in hdfs and exporting it to sqoop table. please find the log details below:

Caused by: java.lang.RuntimeException: Can't parse input data: ' characters'

at tags.__loadFromFields(tags.java:335)

at tags.parse(tags.java:268)

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)

... 10 more

sqoop export command

sqoop export \

--connect "**************************************" \

--username=**** \

--password=***** \

--table tags \

--export-dir /user/cloudera/movie_lens/tags_no_header.csv \

--batch \

--input-lines-terminated-by '\n' \

--input-fields-terminated-by ',' \

--num-mappers 9 \

Table Structure :

create table tags

(userId integer

,movieId integer

,tag varchar(150)

,timestamp decimal

);

Record failing :

660,260,"imaginary world, characters, story, philosophical",1436680217

As per my understanding It's failing because of ambiguous parsing caused by comma ',' in middle of string.

Please help me to understand usage of --input-enclosed-by and --input-escaped-by arguments in this case or Is there any other solution.

解决方案

I have got it resolved using attribute --input-optionally-enclosed-by.

export command :

sqoop export \

--connect "jdbc:mysql://quickstart.cloudera:3306/movie_lens_db" \

--username=root \

--password=cloudera \

--table tags \

--export-dir /user/cloudera/escape_by_test.txt \

--batch \

--input-lines-terminated-by '\n' \

--input-fields-terminated-by ',' \

--input-optionally-enclosed-by '\"' \

--num-mappers 1 \

--outdir java_files

Table Data :

+--------+---------+---------------------------------------------------+------------+

| userId | movieId | tag | timestamp |

+--------+---------+---------------------------------------------------+------------+

| 660 | 260 | imaginary world, characters, story, philosophical | 1436680217 |

| 212 | 69712 | genuine characters | 1260688086 |

+--------+---------+---------------------------------------------------+------------+

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值