sqoop2
版本:sqoop-1.99.6
安装目录:/usr/local/sqoop
参考博文
https://blog.csdn.net/zhexcel/article/details/52192770
https://blog.csdn.net/gamer_gyt/article/details/55225700安装步骤
- 解压软件包
tar -zxvf sqoop-1.99.6-bin-hadoop200.tar.gz -C /usr/local/sqoop
- 配置
# 配置hadoop的proxyuser # core-site.xml中的hadoop.proxyuser.$SERVER_USER.hosts和hadoop.proxyuser.$ SERVER_USER.groups中两个属性 <property> <name>hadoop.proxyuser.sqoop2.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.sqoop2.groups</name> <value>*</value> </property> # 设置sqoop的环境变量 # etc/profile export SQOOP_HOME=/usr/local/sqoop # sqoop使用的第三方组件的路径,此目录需手动创建,mysql的jdbc驱动的jar包需复制到此目录 export SQOOP_SERVER_EXTRA_LIB=$SQOOP_HOME/extra export PATH=$PATH:$SQOOP_HOME/bin # 配置server/conf目录下的sqoop.properties # 路径为hadoop的配置文件目录 org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/etc/hadoop # 配置server/conf目录下的catalina.properties # 将其中引用jar的目录地址改成hadoop中的地址(刚开始没有配置此项,sqoop2-tool verify报错,配置后运行正常) common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop/share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar
初始化(第一次启动时)
# 脚本在$SQOOP_HOME/bin目录下 sqoop2-tool upgrade
验证配置
# 脚本在$SQOOP_HOME/bin目录下 sqoop2-tool verify
启动服务器
# 脚本在$SQOOP_HOME/bin目录下 sqoop.sh server start sqoop2-server start
sqoop1
版本:sqoop-1.4.6
安装目录:/usr/local/sqoop参考博文
https://blog.csdn.net/dwld_3090271/article/details/50747639 https://www.cnblogs.com/mmzs/p/8149921.html
安装步骤
解压软件包
tar -zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /usr/local/sqoop
设置环境变量
# etc/profile # sqoop路径 export SQOOP_HOME=usr/local/sqoop export PATH=$PATH:$SQOOP_HOME
将mysql的jdbc驱动包复制到$SQOOP_HOME/lib目录下
启动hadoop的JobHistoryServer进程
# 在执行导入时,会自动连接JobHistoryServer如果进程没有启动会显示错误信息。 $HADOOP_HOME/sbin/mr-jobhistory-daemon.sh start historyserver
【sqoop】sqoop1和sqoop2环境安装
最新推荐文章于 2024-05-14 08:59:18 发布