SQOOP安装

最新推荐文章于 2024-06-03 12:06:11 发布

XK&RM

最新推荐文章于 2024-06-03 12:06:11 发布

阅读量166

点赞数

分类专栏： Sqoop

本文链接：https://blog.csdn.net/qq_41301707/article/details/103842519

版权

Sqoop 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

SQOOP安装

1.下载SQOOP

[hadoop@xkhadoop software]$ wget http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.16.2.tar.gz

2.解压SQOOP

[hadoop@xkhadoop software]$ tar -rzvf sqoop-1.4.6-cdh5.16.2.tar.gz -C ../app/
[hadoop@xkhadoop software]$ cd ../app/
[hadoop@xkhadoop app]$ ll
总用量 4
drwxrwxrwx.  3 hadoop hadoop   36 1月   4 18:59 erp
lrwxrwxrwx.  1 hadoop hadoop   22 11月 29 22:42 hadoop -> hadoop-2.6.0-cdh5.16.2
drwxr-xr-x. 15 hadoop hadoop  253 11月 29 23:07 hadoop-2.6.0-cdh5.16.2
lrwxrwxrwx.  1 hadoop hadoop   20 12月 16 20:37 hive -> hive-1.1.0-cdh5.16.2
drwxr-xr-x. 12 hadoop hadoop  208 12月 16 22:31 hive-1.1.0-cdh5.16.2
drwxrwxr-x. 10 hadoop hadoop  156 1月   4 20:05 rundeck
drwxr-xr-x. 10 hadoop hadoop 4096 6月   3 2019 sqoop-1.4.6-cdh5.16.2

3.配置SQOOP软连接

[hadoop@xkhadoop app]$ ln -s sqoop-1.4.6-cdh5.16.2 sqoop
[hadoop@xkhadoop app]$ ll
总用量 4
drwxrwxrwx.  3 hadoop hadoop   36 1月   4 18:59 erp
lrwxrwxrwx.  1 hadoop hadoop   22 11月 29 22:42 hadoop -> hadoop-2.6.0-cdh5.16.2
drwxr-xr-x. 15 hadoop hadoop  253 11月 29 23:07 hadoop-2.6.0-cdh5.16.2
lrwxrwxrwx.  1 hadoop hadoop   20 12月 16 20:37 hive -> hive-1.1.0-cdh5.16.2
drwxr-xr-x. 12 hadoop hadoop  208 12月 16 22:31 hive-1.1.0-cdh5.16.2
drwxrwxr-x. 10 hadoop hadoop  156 1月   4 20:05 rundeck
lrwxrwxrwx.  1 hadoop hadoop   21 1月   5 16:14 sqoop -> sqoop-1.4.6-cdh5.16.2
drwxr-xr-x. 10 hadoop hadoop 4096 1月   5 16:13 sqoop-1.4.6-cdh5.16.2

四.修改配置文件及配置环境变量

[hadoop@xkhadoop sqoop]$ cd conf/
[hadoop@xkhadoop conf]$ ll
总用量 28
-rw-rw-r--. 1 hadoop hadoop 3895 6月   3 2019 oraoop-site-template.xml
-rw-rw-r--. 1 hadoop hadoop 1404 6月   3 2019 sqoop-env-template.cmd
-rwxr-xr-x. 1 hadoop hadoop 1345 6月   3 2019 sqoop-env-template.sh
-rw-rw-r--. 1 hadoop hadoop 6044 6月   3 2019 sqoop-site-template.xml
-rw-rw-r--. 1 hadoop hadoop 6044 6月   3 2019 sqoop-site.xml
[hadoop@xkhadoop conf]$ cp sqoop-env-template.sh sqoop-env.sh
[hadoop@xkhadoop conf]$ vim sqoop-env.sh 
export HADOOP_COMMON_HOME=/home/hadoop/app/hadoop
export HADOOP_MAPRED_HOME=/home/hadoop/app/hadoop
export HIVE_HOME=/home/hadoop/app/hive

[hadoop@xkhadoop conf]$ ll
总用量 32
-rw-rw-r--. 1 hadoop hadoop 3895 6月   3 2019 oraoop-site-template.xml
-rwxr-xr-x. 1 hadoop hadoop 1345 1月   5 16:16 sqoop-env.sh
-rw-rw-r--. 1 hadoop hadoop 1404 6月   3 2019 sqoop-env-template.cmd
-rwxr-xr-x. 1 hadoop hadoop 1345 6月   3 2019 sqoop-env-template.sh
-rw-rw-r--. 1 hadoop hadoop 6044 6月   3 2019 sqoop-site-template.xml
-rw-rw-r--. 1 hadoop hadoop 6044 6月   3 2019 sqoop-site.xml
[hadoop@xkhadoop conf]$ cd ~
[hadoop@xkhadoop ~]$ vim .bashrc
export SQOOP_HOME=/home/hadoop/app/sqoop
export PATH=$SQOOP_HOME/bin:$PATH

[root@xkhadoop ~]# find / -name '*mysql-connector-java*'
/home/hadoop/app/hive-1.1.0-cdh5.16.2/lib/mysql-connector-java-5.1.27.jar
[hadoop@xkhadoop lib]$ cp /home/hadoop/app/hive-1.1.0-cdh5.16.2/lib/mysql-connector-java-5.1.27.jar ./
[hadoop@xkhadoop lib]$ ll
总用量 31344
-rw-rw-r--. 1 hadoop hadoop   224277 6月   3 2019 ant-contrib-1.0b3.jar
-rw-rw-r--. 1 hadoop hadoop    36455 6月   3 2019 ant-eclipse-1.0-jvm1.2.jar
-rw-rw-r--. 1 hadoop hadoop   477209 6月   3 2019 avro-1.7.6-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   181328 6月   3 2019 avro-mapred-1.7.6-cdh5.16.2-hadoop2.jar
-rw-rw-r--. 1 hadoop hadoop    58160 6月   3 2019 commons-codec-1.4.jar
-rw-rw-r--. 1 hadoop hadoop   425643 6月   3 2019 commons-compress-1.11.jar
-rw-rw-r--. 1 hadoop hadoop   109043 6月   3 2019 commons-io-1.4.jar
-rw-rw-r--. 1 hadoop hadoop   267634 6月   3 2019 commons-jexl-2.1.1.jar
-rw-rw-r--. 1 hadoop hadoop   434678 6月   3 2019 commons-lang3-3.4.jar
-rw-rw-r--. 1 hadoop hadoop    62050 6月   3 2019 commons-logging-1.1.3.jar
-rw-rw-r--. 1 hadoop hadoop 14582557 6月   3 2019 fastutil-6.3.jar
-rw-rw-r--. 1 hadoop hadoop   706710 6月   3 2019 hsqldb-1.8.0.10.jar
-rw-rw-r--. 1 hadoop hadoop    36519 6月   3 2019 jackson-annotations-2.3.1.jar
-rw-rw-r--. 1 hadoop hadoop   197986 6月   3 2019 jackson-core-2.3.1.jar
-rw-rw-r--. 1 hadoop hadoop   227535 6月   3 2019 jackson-core-asl-1.8.10.jar
-rw-rw-r--. 1 hadoop hadoop   917322 6月   3 2019 jackson-databind-2.3.1-cloudera.2.jar
-rw-rw-r--. 1 hadoop hadoop   672097 6月   3 2019 jackson-mapper-asl-1.8.10-cloudera.2.jar
-rw-rw-r--. 1 hadoop hadoop  2201079 6月   3 2019 kite-data-core-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop  1808186 6月   3 2019 kite-data-hive-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop  1773706 6月   3 2019 kite-data-mapreduce-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop  1772115 6月   3 2019 kite-hadoop-compatibility-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   872300 1月   5 16:37 mysql-connector-java-5.1.27.jar
-rw-rw-r--. 1 hadoop hadoop    19827 6月   3 2019 opencsv-2.3.jar
-rw-rw-r--. 1 hadoop hadoop    29555 6月   3 2019 paranamer-2.3.jar
-rw-rw-r--. 1 hadoop hadoop   106454 6月   3 2019 parquet-avro-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   956036 6月   3 2019 parquet-column-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop    41085 6月   3 2019 parquet-common-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   278926 6月   3 2019 parquet-encoding-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   384616 6月   3 2019 parquet-format-2.1.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   212649 6月   3 2019 parquet-hadoop-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop   931451 6月   3 2019 parquet-jackson-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop    26084 6月   3 2019 slf4j-api-1.7.5.jar
-rw-rw-r--. 1 hadoop hadoop   995968 6月   3 2019 snappy-java-1.0.4.1.jar

五.测试下sqoop是否可以使用

[hadoop@xkhadoop lib]$ sqoop help
Warning: /home/hadoop/app/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/hadoop/app/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/app/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/hadoop/app/sqoop/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/01/05 16:37:37 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.2
usage: sqoop COMMAND [ARGS]

Available commands:
  codegen            Generate code to interact with database records
  create-hive-table  Import a table definition into Hive
  eval               Evaluate a SQL statement and display the results
  export             Export an HDFS directory to a database table
  help               List available commands
  import             Import a table from a database to HDFS
  import-all-tables  Import tables from a database to HDFS
  import-mainframe   Import datasets from a mainframe server to HDFS
  job                Work with saved jobs
  list-databases     List available databases on a server
  list-tables        List available tables in a database
  merge              Merge results of incremental imports
  metastore          Run a standalone Sqoop metastore
  version            Display version information

See 'sqoop help COMMAND' for information on a specific command.