下载文件
wget http://mirrors.hust.edu.cn/apache/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
解压到/opt/目录下
tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /opt/
设置Sqoop环境变量
vim /etc/profile
export SQOOP_HOME=/opt/sqoop-1.4.7.bin__hadoop-2.6.0
export PATH=$PATH:$SQOOP_HOME/bin
source /etc/profile
配置Sqoop用Hadoop,需要编辑 sqoop-env.sh 文件,该文件被放置在 $SQOOP_HOME/conf 目录。首先要重定向到 Sqoop conf 目录,并使用以下命令复制的模板文件
cd $SQOOP_HOME/conf
mv sqoop-env-template.sh sqoop-env.sh
vim sqoop-env.sh
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/opt/hadoop-2.9.1
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/opt/hadoop-2.9.1
下载数据库驱动到lib目录
因为我用的是mysql,所以我就下载mysql驱动到lib
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.40/mysql-connector-java-5.1.40.jar
在将mysql数据导入到hive中,遇到了一个错误
ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
解决办法
将Hive lib下面的jar包拷贝到SQOOP的lib下
cp hive-common-2.3.3.jar $SQOOP_HOME/lib
cp hive-shims* $SQOOP_HOME/lib