sqoop导出数据到mysql程序长时间执行未完成，且mr控制台也没有报错，如何排查

1.当sqoop导出程序长时间执行未完成，且mr控制台也没有报错

[w4xj@node202 sqoop-1.4.6.bin__hadoop-2.0.4-alpha]$ bin/sqoop export \

> --connect jdbc:mysql://node202:3306/company \

> --username root \

> --password 123456 \

> --table staff \

> --num-mappers 1 \

> --export-dir /user/hive/warehouse/staff_hive \

> --input-fields-terminated-by "\t"

Warning: /opt/module/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hbase does not exist! HBase imports will fail.

Please set $HBASE_HOME to the root of your HBase installation.

Warning: /opt/module/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: /opt/module/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

19/03/31 11:02:55 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

19/03/31 11:02:55 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

19/03/31 11:02:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

19/03/31 11:02:55 INFO tool.CodeGenTool: Beginning code generation

19/03/31 11:02:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `staff` AS t LIMIT 1

19/03/31 11:02:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `staff` AS t LIMIT 1

19/03/31 11:02:56 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/module/hadoop-2.7.2

注: /tmp/sqoop-w4xj/compile/299e764abb8e42f075ad01d3ab61c92f/staff.java使用或覆盖了已过时的 API。

注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。

19/03/31 11:02:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-w4xj/compile/299e764abb8e42f075ad01d3ab61c92f/staff.jar

19/03/31 11:02:57 INFO mapreduce.ExportJobBase: Beginning export of staff

19/03/31 11:02:57 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar

19/03/31 11:02:58 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative

19/03/31 11:02:58 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative

19/03/31 11:02:58 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

19/03/31 11:02:58 INFO client.RMProxy: Connecting to ResourceManager at node203/192.168.26.203:8032

19/03/31 11:03:01 INFO input.FileInputFormat: Total input paths to process : 1

19/03/31 11:03:01 INFO input.FileInputFormat: Total input paths to process : 1

19/03/31 11:03:01 INFO mapreduce.JobSubmitter: number of splits:1

19/03/31 11:03:01 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative

19/03/31 11:03:01 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1553997792849_0005

19/03/31 11:03:01 INFO impl.YarnClientImpl: Submitted application application_1553997792849_0005

19/03/31 11:03:01 INFO mapreduce.Job: The url to track the job: http://node203:8088/proxy/application_1553997792849_0005/

19/03/31 11:03:01 INFO mapreduce.Job: Running job: job_1553997792849_0005

19/03/31 11:03:12 INFO mapreduce.Job: Job job_1553997792849_0005 running in uber mode : false

19/03/31 11:03:12 INFO mapreduce.Job:  map 0% reduce 0%

19/03/31 11:03:25 INFO mapreduce.Job:  map 100% reduce 0%

19/03/31 11:13:40 INFO mapreduce.Job: Task Id : attempt_1553997792849_0005_m_000000_0, Status : FAILED

AttemptID:attempt_1553997792849_0005_m_000000_0 Timed out after 600 secs

19/03/31 11:13:42 INFO mapreduce.Job:  map 0% reduce 0%

19/03/31 11:13:53 INFO mapreduce.Job:  map 100% reduce 0%

2.那么我们打开yarn的控制台http://node203:8088/cluster，找到那个正在执行的job，点击ApplicationMaster

3.点击Job_ID，查看当前job的执行情况

4.点击正在运行的map

5.点击任务

6.点击logs

7.点击syslog

8.我们可以看到mysql实际已经报错了，但是sqoop控制台并没有报错，等待最终job失败才返回job失败

9.关于返回错误

[1].返回错误：

19/03/31 11:45:12 INFO mapreduce.Job: Job job_1553997792849_0005 failed with state FAILED due to: Task failed task_1553997792849_0005_m_000000

Job failed as tasks failed. failedMaps:1 failedReduces:0





19/03/31 11:45:12 INFO mapreduce.Job: Counters: 9

        Job Counters

                Failed map tasks=4

                Launched map tasks=4

                Other local map tasks=3

                Rack-local map tasks=1

                Total time spent by all maps in occupied slots (ms)=2512919

                Total time spent by all reduces in occupied slots (ms)=0

                Total time spent by all map tasks (ms)=2512919

                Total vcore-milliseconds taken by all map tasks=2512919

                Total megabyte-milliseconds taken by all map tasks=2573229056

19/03/31 11:45:12 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead

19/03/31 11:45:12 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 2,534.5903 seconds (0 bytes/sec)

19/03/31 11:45:12 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead

19/03/31 11:45:12 INFO mapreduce.ExportJobBase: Exported 0 records.

19/03/31 11:45:12 ERROR tool.ExportTool: Error during export: Export job failed!

[2].若不返回错误，程序一直卡在那儿，按Ctrl C 停止后，job还在继续运行，需要在ResourceManager上手动停止job，这里的jobId就是上面的那个Job ID

sh yarn application -kill jobId