sqoop hive导出mysql_使用 sqoop 将 hive 数据导出到 mysql (export)

这篇博客介绍了如何使用 Sqoop 工具将 Hive 中的数据导出到 MySQL 数据库。通过执行 Sqoop 命令,连接 MySQL 数据库,设置用户名、密码、目标表名及 Hive 的外部表目录,成功完成数据导出。过程中遇到了 SLF4J 多绑定警告,但不影响任务执行。最终, Sqoop 成功地将 3 条记录从 Hive 导出到 MySQL,并在 26.9573 秒内完成了 758 字节的数据传输。
摘要由CSDN通过智能技术生成

EFdeMacBook-Pro:bin FengZhen$ sqoop export --connect jdbc:mysql://localhost:3306/sqooptest --username root --password 123qwe --table sqoop_test --export-dir /user/hive/external/sqoop_test --input-fields-terminated-by ,

Warning: /Users/FengZhen/Desktop/Hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/../hcatalog does not exist!HCatalog jobs will fail.

Pleaseset$HCAT_HOME to the root of your HCatalog installation.

Warning:/Users/FengZhen/Desktop/Hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/../accumulo does not exist!Accumulo imports will fail.

Pleaseset$ACCUMULO_HOME to the root of your Accumulo installation.

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found bindingin [jar:file:/Users/FengZhen/Desktop/Hadoop/hadoop-2.8.0/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found bindingin [jar:file:/Users/FengZhen/Desktop/Hadoop/hbase-1.3.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding isof type [org.slf4j.impl.Log4jLoggerFactory]17/09/13 10:28:01 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

17/09/13 10:28:01 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.17/09/13 10:28:01INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.17/09/13 10:28:01INFO tool.CodeGenTool: Beginning code generation17/09/13 10:28:02 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `sqoop_test` AS t LIMIT 1

17/09/13 10:28:02 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `sqoop_test` AS t LIMIT 1

17/09/13 10:28:02 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /Users/FengZhen/Desktop/Hadoop/hadoop-2.8.0

17/09/13 10:28:04 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-FengZhen/compile/7a078053fb0424d718e08c56fc9bab27/sqoop_test.jar17/09/13 10:28:04INFO mapreduce.ExportJobBase: Beginning export of sqoop_test17/09/13 10:28:04 INFO Configuration.deprecation: mapred.job.tracker isdeprecated. Instead, use mapreduce.jobtracker.address17/09/13 10:28:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes whereapplicable17/09/13 10:28:04 INFO Configuration.deprecation: mapred.jar isdeprecated. Instead, use mapreduce.job.jar17/09/13 10:28:05 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution isdeprecated. Instead, use mapreduce.reduce.speculative17/09/13 10:28:05 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution isdeprecated. Instead, use mapreduce.map.speculative17/09/13 10:28:05 INFO Configuration.deprecation: mapred.map.tasks isdeprecated. Instead, use mapreduce.job.maps17/09/13 10:28:06 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8032

17/09/13 10:28:07 INFO input.FileInputFormat: Total input files to process : 1

17/09/13 10:28:07 INFO input.FileInputFormat: Total input files to process : 1

17/09/13 10:28:07 INFO mapreduce.JobSubmitter: number of splits:4

17/09/13 10:28:07 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution isdeprecated. Instead, use mapreduce.map.speculative17/09/13 10:28:07 INFO mapreduce.JobSubmitter: Submitting tokens forjob: job_1505268150495_000417/09/13 10:28:08INFO impl.YarnClientImpl: Submitted application application_1505268150495_000417/09/13 10:28:08 INFO mapreduce.Job: The url to track the job: http://192.168.1.64:8088/proxy/application_1505268150495_0004/

17/09/13 10:28:08INFO mapreduce.Job: Running job: job_1505268150495_000417/09/13 10:28:18 INFO mapreduce.Job: Job job_1505268150495_0004 running in uber mode : false

17/09/13 10:28:18 INFO mapreduce.Job: map 0% reduce 0%

17/09/13 10:28:32 INFO mapreduce.Job: map 100% reduce 0%

17/09/13 10:28:32INFO mapreduce.Job: Job job_1505268150495_0004 completed successfully17/09/13 10:28:32 INFO mapreduce.Job: Counters: 30File System Counters

FILE: Number of bytes read=0FILE: Number of bytes written=626576FILE: Number of read operations=0FILE: Number of large read operations=0FILE: Number of write operations=0HDFS: Number of bytes read=758HDFS: Number of bytes written=0HDFS: Number of read operations=19HDFS: Number of large read operations=0HDFS: Number of write operations=0Job Counters

Launched map tasks=4Data-local map tasks=4Total time spent by all mapsin occupied slots (ms)=45180Total time spent by all reducesin occupied slots (ms)=0Total time spent by all map tasks (ms)=45180Total vcore-milliseconds taken by all map tasks=45180Total megabyte-milliseconds taken by all map tasks=46264320Map-Reduce Framework

Map input records=3Map output records=3Input split bytes=671Spilled Records=0Failed Shuffles=0Merged Map outputs=0GC time elapsed (ms)=266CPU time spent (ms)=0Physical memory (bytes) snapshot=0Virtual memory (bytes) snapshot=0Total committed heap usage (bytes)=599785472File Input Format Counters

Bytes Read=0File Output Format Counters

Bytes Written=0

17/09/13 10:28:32 INFO mapreduce.ExportJobBase: Transferred 758 bytes in 26.9573 seconds (28.1185 bytes/sec)17/09/13 10:28:32 INFO mapreduce.ExportJobBase: Exported 3 records.

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值