是一个命令行工具?
使得结构化数据与Hadoop之间的导入导出变得容易
可以导入整个库或者单个表
可以与Oozie集成,将 导入导出作为工作流的一部分。
内部转化为MapReduce,用于传输数据
下载安装配置
根据Hadoop版本选择对应的Sqoop,我这里是Hadoop,选择Sqoop 1.99.6:
wget http://mirrors.hust.edu.cn/apache/sqoop/1.99.6/sqoop-1.99.6-bin-hadoop200.tar.gztar zxvf sqoop-1.99.6-bin-hadoop200.tar.gz mv sqoop-1.99.6-bin-hadoop200 /home/sqoop-1.99.612345
配置PATH:
vim /etc/profileexport SQOOP_HOME=/home/sqoop-1.99.6export PATH=$PATH:$SQOOP_HOME/binsource /etc/profile12345
配置server/conf/catalina.properties
common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/home/hadoop-2.6.0/share/hadoop/common/*.jar,/home/hadoop-2.6.0/share/hadoop/common/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/hdfs/*.jar,/home/hadoop-2.6.0/share/hadoop/hdfs/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/mapreduce/*.jar,/home/hadoop-2.6.0/share/hadoop/mapreduce/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/yarn/*.jar,/home/hadoop-2.6.0/share/hadoop/yarn/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/tools/*.jar,/home/hadoop-2.6.0/share/hadoop/tools/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/httpfs/tomcat/lib/*.jar 1
配置server/conf/sqoop.properties:
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop-2.6.0/etc/hadoop1
验证是否配置正确:
sqoop2-tool veriify1
启动sqoop server:
% sqoop2-server start1
启动的的tomcat运行在12000端口。
启动客户端shell:
sqoop2-shell1
Sqoop实例
启动Hadoop集群:
start-dfs.shstart-yarn.sh123
启动sqoop2-shell:
sqoop2-shell1
查看版本和connector:
show version -allshow connector12
创建JDBC link: