一、Sqoop将Mysql数据导入到HDFS
Shell:实例
sqoop import -m 1 --connect jdbc:mysql://192.168.1.1:3306/elens_oauth --username root --password 123456 --table rbc_traffic_radio_user_bak --target-dir /rbc_source/Traffic_program/rbc_traffic_radio_user -fields-terminated-by "^" --hive-drop-import-delims --delete-target-dir
192.168.1.1:3306 :为关系型数据库IP:端口
elens_oauth :为数据库的库
root 123456 :分别为用户名密码! eg:密码有特殊字符需要转义 : \!
rbc_traffic_radio_user_bak :为表名
/rbc_source/Traffic_program/rbc_traffic_radio_user :为数据导入到HDFS路径地址
fields-terminated-by "^" :数据字段分割为 ^
--hive-drop-import-delims --delete-target-dir :为每次数据导入时,删除目标文件夹
sqoop import -m 1 --connect jdbc:mysql://39.105.209.116:3306/rbc_wechat_data --username root --password server123\!@# --table rbc_traffic_radio_user_bak --target-dir /test_rbcdata/rbc_traffic_radio_user -fields-terminated-by "^" --hive-drop-import-delims --delete-target-dir
二、Sqoop将HDFS数据导入到Mysql
Shell:实例
sqoop export --connect "jdbc:mysql://192.168.1.1:3306/elens_oauth?useUnicode=true&characterEncoding=utf-8" --username root --password 123456 --table rbc_ttfm_album_audios --export-dir /rbc_source/tingtingfm/rbc_ttfm_album_audios/ --input-fields-terminated-by '^'
useUnicode=true&characterEncoding=utf-8 :设置编码格式为UTF-8
--table rbc_ttfm_album_audios :数据导入的mysql表为rbc_ttfm_album_audios
--export-dir /rbc_source/tingtingfm/rbc_ttfm_album_audios/ :为HDFS数据存放的地址
--input-fields-terminated-by '^' :为数据字段分割为 ^
sqoop export --connect "jdbc:mysql://39.105.209.116:3306/elens_oauth?useUnicode=true&characterEncoding=utf-8" --username root --password server123\!@# --table rbc_ttfm_album_audios --export-dir /rbc_source/tingtingfm/rbc_ttfm_album_audios/ --input-fields-terminated-by '^'
如果不设置编码格式即为:
sqoop export --connect jdbc:mysql://192.168.1.1:3306/elens_oauth --username root --password 123456 --table rbc_ttfm_album_audios --export-dir /rbc_source/tingtingfm/rbc_ttfm_album_audios/ --input-fields-terminated-by '^'