sqoop安装与验证

解压

将sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz拷贝到linux下，然后解压

tar –zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

然后进入新生成的目录

cd sqoop-1.4.6.bin__hadoop-2.0.4-alpha

cd /conf

cp sqoop-env-template.sh sqoop-env.sh

修改sqoop-env.sh(下面的目录换成你自己的安装目录)

#Set path to where bin/hadoop is available export HADOOP_COMMON_HOME=/usr/tools/hadoop-2.6.4   #Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/usr/tools/hadoop-2.6.4   #set the path to where bin/hbase is available export HBASE_HOME=/usr/tools/hbase-1.2.0   #Set the path to where bin/hive is available export HIVE_HOME=/usr/tools/apache-hive-2.0.0-bin   #Set the path for where zookeper config dir is export ZOOCFGDIR=/usr/zookeeper-3.4.5

修改环境变量：

vi /etc/profile

#sqoop

export SQOOP_HOME=/usr/tools/sqoop-1.4.6.bin__hadoop-2.0.4-alpha

export PATH=$PATH:$SQOOP_HOME/bin

保存后使其生效

source /etc/profile

检查sqoop是否安装成功：sqoop help

简单测试：

开启mysql远程访问：

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123qwe';

FLUSH PRIVILEGES;

列出链接数据所有表

sqoop list-tables -connect jdbc:mysql://localhost:3306/test --username root --password 123qwe

这一步可能会报错： Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 20/05/15 17:16:25 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 20/05/15 17:16:25 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 20/05/15 17:16:25 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 20/05/15 17:16:25 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:875)        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)        at org.apache.sqoop.manager.CatalogQueryManager.listTables(CatalogQueryManager.java:102)        at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49)        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)        at org.apache.sqoop.Sqoop.main(Sqoop.java:252) 原因：/opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/lib目录下缺少mysql的jar驱动 解决办法：把hive的lib下的mysql-connector-java-5.1.47.jar复制到sqoop的lib目录下即可

查看结果与MySQL的show tables;结果一致

注意，这里会报警告：

Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

这主要是因为集群中没有安装HCatal和accumulo环境变量，但sqoop默认会检测这两个组件。若非要去掉这两个检测，可以修改bin/config-sqoop文件，很容易找到这两个组件的检测位置。把下面两行注释掉即可：（你也可以选择安装HCatal和accumulo这两个组件）