关闭

hadoop集群sqoop2安装指南

标签: hadoopsqoopubuntu大数据
964人阅读 评论(0) 收藏 举报
分类:

环境说明

操作系统:Ubuntu 14.04

集群节点:

  • Master
  • slave1
  • slave2

解压并安装

hadoop@Master:~$ sudo tar xvfz sqoop-1.99.6-bin-hadoop200.tar.gz 
hadoop@Master:~$ sudo mv sqoop-1.99.6-bin-hadoop200 /usr/local/sqoop
hadoop@Master:~$ sudo chmod -R 775 /usr/local/sqoop
hadoop@Master:~$ sudo chown -R hadoop:hadoop /usr/local/sqoop

修改环境变量

hadoop@Master:~$ sudo nano /etc/profile

#sqoop
export SQOOP_HOME=/usr/local/sqoop
export PATH=$SQOOP_HOME/bin:$PATH
export CATALINA_BASE=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs

hadoop@Master:~$ source /etc/profile

修改sqoop的环境变量

hadoop@Master:/$ sudo nano /usr/local/sqoop/server/conf/sqoop.properties 

#修改指向我的hadoop安装目录  
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/etc/hadoop  

hadoop@Master:/$ sudo nano /usr/local/sqoop/server/conf/catalina.properties 

common.loader=/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop/share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop/share/hadoop/httpfs/tomcat/lib/*.jar,

下载mysql驱动包,mysql-connector-java-5.1.27.jar

把jar包丢到到$SQOOP_HOME/server/lib下面

$ sudo cp mysql-connector-java-5.1.27.jar $SQOOP_HOME/server/lib

有时,启动sqoop时可能会遇到找不到JAVA_HOME的情况,为了保险起见我们直接在配置文件中写入JAVA_HOME

#在/usr/local/sqoop/server/bin/setenv.sh中,添加
export JAVA_HOME=/usr/lib/jvm/

启动sqoop

hadoop@Master:~/mysql-connector-java-5.0.8$ sqoop.sh server start
Sqoop home directory: /usr/local/sqoop
Setting SQOOP_HTTP_PORT:     12000
Setting SQOOP_ADMIN_PORT:     12001
Using   CATALINA_OPTS:       
Adding to CATALINA_OPTS:    -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE:   /usr/local/sqoop/server
Using CATALINA_HOME:   /usr/local/sqoop/server
Using CATALINA_TMPDIR: /usr/local/sqoop/server/temp
Using JRE_HOME:        /usr/lib/jvm//jre
Using CLASSPATH:       /usr/local/sqoop/server/bin/bootstrap.jar

验证启动成功

如果服务端已经启动,则有Bootstrap这个进程:

hadoop@Master:~$ jps
...
13771 Bootstrap
...
1
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:32818次
    • 积分:637
    • 等级:
    • 排名:千里之外
    • 原创:25篇
    • 转载:0篇
    • 译文:5篇
    • 评论:1条
    文章分类
    最新评论