sqoop
文章平均质量分 85
Lyle-liang
大数据,互联网金融交易
展开
-
Sqoop导数据 S3 to Mysql
使用Sqoop导数据,步骤:1. 在mysql中建表2.将s3数据copy到hdfs:hadoop distcp s3n://dir-path hdfs://dir-path3.用sqoop将hdfs中的数据导入mysql:sqoop export \--connect 'jdbc:mysql://mysqlHost:3306/dbName' \--username 'xxx' \--passwo...原创 2018-05-01 20:27:20 · 1067 阅读 · 0 评论 -
sqoop 从hdfs向TiDB导数据,WriteConflict
问题:java.io.IOException: java.sql.SQLException: (conn=16679) [try again later]: tikv restarts txn: Txn(Mvcc(WriteConflict))待解决:原创 2018-05-02 11:22:41 · 2527 阅读 · 0 评论 -
AWS oozie sqoop action 调度mysql to hive
本文基于AWS S3,oozie 4.5.0,sqoop 1.4.7,sqoop自己安装,其余AWS安装。配置SQOOP_HOME环境变量vim /etc/profileexport SQOOP_HOME=/usr/lib/sqoopexport PATH=$PATH:$SQOOP_HOME/bin配置sqoop-env.sh# included in all the hadoop scripts...原创 2018-05-10 18:52:33 · 559 阅读 · 0 评论 -
【Sqoop】Descriptor location does not exist: hdfs://<path>.metadata
Sqoop v1.4.6 Bug,目前还在修复,fix version待定:https://issues.apache.org/jira/browse/SQOOP-3151原创 2018-05-13 19:25:49 · 1556 阅读 · 0 评论