简介
Sqoop是一个用于在外部结构化数据与Hadoop之间导入导出数据的工具。
Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases.
Sqoop:
- 是一个命令行工具?
- 使得结构化数据与Hadoop之间的导入导出变得容易
- 可以导入整个库或者单个表
- 可以与Oozie集成,将 导入导出作为工作流的一部分。
- 内部转化为MapReduce,用于传输数据
下载安装配置
根据Hadoop版本选择对应的Sqoop,我这里是Hadoop,选择Sqoop 1.99.6:
wget http://mirrors.hust.edu.cn/apache/sqoop/1.99.6/sqoop-1.99.6-bin-hadoop200.tar.gz
tar zxvf sqoop-1.99.6-bin-hadoop200.tar.gz
mv sqoop-1.99.6-bin-hadoop200 /home/sqoop-1.99.6
配置PATH:
vim /etc/profile
export SQOOP_HOME=/home/sqoop-1.99.6
export PATH=$PATH:$SQOOP_HOME/bin
source /etc/profile
配置server/conf/catalina.properties
common.loader=${
catalina.base}/lib,${
catalina.base}/lib/*.jar,${
catalina.home}/lib,${
catalina.home}/lib/*.jar,${
catalina.home}/../lib/*.jar,/home/hadoop-2.6.0/share/hadoop/common/*.jar,/home/hadoop-2.6.0/share/hadoop/common/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/hdfs/*.jar,/home/hadoop-2.6.0/share/hadoop/hdfs/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/mapreduce/*.jar,/home/hadoop-2.6.0/share/hadoop/mapreduce/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/yarn/*.jar,/home/hadoop-2.6.0/share/hadoop/yarn/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/tools/*.jar,/home/hadoop-2.6.0/share/hadoop/tools/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/httpfs/tomcat/lib/*.jar
配置server/conf/sqoop.properties:
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop-2.6.0/etc/hadoop
验证是否配置正确:
sqoop2-tool veriify
启动sqoop serv