sqoop同步hdfs与mysql端口_使用Sqoop将数据在HDFS与MySQL互导

1.去官网下载sqoop，直接百度即可

2.解压后进入conf目录

guo@drguo1:/opt/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf$ cp sqoop-env-template.sh sqoop-env.sh

3.在sqoop-env.sh添加各种home

#Set path to where bin/hadoop is available

export HADOOP_COMMON_HOME=/opt/Hadoop/hadoop-2.7.2

#Set path to where hadoop-*-core.jar is available

export HADOOP_MAPRED_HOME=/opt/Hadoop/hadoop-2.7.2

#set the path to where bin/hbase is available

export HBASE_HOME=/opt/hbase

#Set the path to where bin/hive is available

export HIVE_HOME=/opt/apache-hive-2.0.0-bin

#Set the path for where zookeper config dir is

export ZOOCFGDIR=/opt/zookeeper-3.4.8/conf

4.把MySQL驱动jar包导入lib/下

5.建表

mysql> use hive;

Database changed

mysql> create table urlrule(

-> url varchar(1024),

-> info varchar(50)

-> );

Query OK, 0 rows affected (0.12 sec)

mysql> show tables;

+---------------+

| Tables_in_hive |

+---------------+

| urlrule |

+---------------+

1 row in set (0.00 sec)

6.将数据从HDFS导入MySQL

sqoop export --connect jdbc:mysql://drguo1:3306/url --username hive --password guo \

--table urlrule \

--export-dir /flow/topkurl/ \

--columns url \

--input-fields-terminated-by '\t'

7.将数据从MySQL导入HDFS

[hdfs@ws1dn1 root]$ sqoop import -m 1 --connect jdbc:mysql://192.168.1.65:3306/v3 --username root --password guo --table t_log_2016_done --target-dir /user/hdfs/testdata/