mysql传到hdfs需要改格式吗_Sqoop将MySQL的表数据同步到HDFS（四）设置存储格式+压缩...

[root@centos02 bin]#sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 2 --target-dir /jdbcHDFS/TAdminLog_avro_snappy -- as-avrodatafile --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec --fields-terminated-by '\t'

Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hbase does not exist!HBase imports will fail.

Please set$HBASE_HOMEto the root of your HBase installation.

Warning:/opt/bigdata/sqoop/sqoop-1.4.7/../hcatalog does not exist!HCatalog jobs will fail.

Please set$HCAT_HOMEto the root of your HCatalog installation.

Warning:/opt/bigdata/sqoop/sqoop-1.4.7/../accumulo does not exist!Accumulo imports will fail.

Please set$ACCUMULO_HOMEto the root of your Accumulo installation.

Warning:/opt/bigdata/sqoop/sqoop-1.4.7/../zookeeper does not exist!Accumulo imports will fail.

Please set$ZOOKEEPER_HOMEto the root of your Zookeeper installation.19/09/04 01:47:48 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7

19/09/04 01:47:48 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.19/09/04 01:47:49 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.19/09/04 01:47:49 INFO manager.SqlManager: Using default fetchSize of 1000

19/09/04 01:47:49INFO tool.CodeGenTool: Beginning code generation19/09/04 01:47:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0

19/09/04 01:47:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0

19/09/04 01:47:50 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/bigdata/hadoop/hadoop-2.8.5注:/tmp/sqoop-root/compile/1868b99825c89433a0b1b814c04d1e86/tadminoperationlog.java使用或覆盖了已过时的 API。

注: 有关详细信息, 请使用-Xlint:deprecation 重新编译。19/09/04 01:47:53 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/1868b99825c89433a0b1b814c04d1e86/tadminoperationlog.jar19/09/04 01:47:53INFO mapreduce.ImportJobBase: Beginning import of tadminoperationlog19/09/04 01:47:54INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar19/09/04 01:47:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0

19/09/04 01:47:55INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps19/09/04 01:47:56 INFO client.RMProxy: Connecting to ResourceManager at centos02/192.168.122.1:8032

19/09/04 01:48:02INFO db.DBInputFormat: Using read commited transaction isolation19/09/04 01:48:02INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(FID), MAX(FID) FROM tadminoperationlog19/09/04 01:48:02 INFO db.IntegerSplitter: Split size: 16058; Num splits: 2 from: 21 to: 32138

19/09/04 01:48:02 INFO mapreduce.JobSubmitter: number of splits:2

19/09/04 01:48:02 INFO mapreduce.JobSubmitter: Submitting tokens forjob: job_1567503661837_001119/09/04 01:48:03INFO impl.YarnClientImpl: Submitted application application_1567503661837_001119/09/04 01:48:03 INFO mapreduce.Job: The url to track the job: http://centos02:8088/proxy/application_1567503661837_0011/

19/09/04 01:48:03INFO mapreduce.Job: Running job: job_1567503661837_001119/09/04 01:48:16 INFO mapreduce.Job: Job job_1567503661837_0011 running in uber mode : false

19/09/04 01:48:16 INFO mapreduce.Job: map 0% reduce 0%

19/09/04 01:48:26 INFO mapreduce.Job: map 50% reduce 0%

19/09/04 01:48:28 INFO mapreduce.Job: map 100% reduce 0%

19/09/04 01:48:28INFO mapreduce.Job: Job job_1567503661837_0011 completed successfully19/09/04 01:48:28 INFO mapreduce.Job: Counters: 30File System Counters

FILE: Number of bytes read=0FILE: Number of bytes written=357744FILE: Number of read operations=0FILE: Number of large read operations=0FILE: Number of write operations=0HDFS: Number of bytes read=206HDFS: Number of bytes written=3753700HDFS: Number of read operations=8HDFS: Number of large read operations=0HDFS: Number of write operations=4Job Counters

Launched map tasks=2Other local map tasks=2Total time spent by all mapsin occupied slots (ms)=17042Total time spent by all reducesin occupied slots (ms)=0Total time spent by all map tasks (ms)=17042Total vcore-milliseconds taken by all map tasks=17042Total megabyte-milliseconds taken by all map tasks=17451008Map-Reduce Framework

Map input records=12122Map output records=12122Input split bytes=206Spilled Records=0Failed Shuffles=0Merged Map outputs=0GC time elapsed (ms)=1034CPU time spent (ms)=7230Physical memory (bytes) snapshot=364994560Virtual memory (bytes) snapshot=4245737472Total committed heap usage (bytes)=196083712File Input Format Counters

Bytes Read=0File Output Format Counters

Bytes Written=3753700

19/09/04 01:48:28 INFO mapreduce.ImportJobBase: Transferred 3.5798 MB in 33.0098 seconds (111.0494 KB/sec)19/09/04 01:48:28 INFO mapreduce.ImportJobBase: Retrieved 12122records.

[root@centos02 bin]#