hdfs mysql sqoop 失败_求助， sqoop 导出 hive 表到 mysql 出错！

为了调试，mysql 表结构已经简化了，没有任何约束：

mysql> desc tb_stu_answer;

+-----------+----------+------+-----+---------+-------+

| Field | Type | Null | Key | Default | Extra |

+-----------+----------+------+-----+---------+-------+

| sa_id | tinytext | YES | | NULL | |

| sa_stu_id | tinytext | YES | | NULL | |

+-----------+----------+------+-----+---------+-------+

sqoop 脚本如下：

sqoop export --connect jdbc:mysql://xxx:3306/db_name --username xxx --P --table tb_stu_answer --export-dir 'hdfs://aly-bigdata-hadoop-xxx/user/hive/warehouse/xx.db/tb_stu_answer_test' --input-fields-terminated-by '\001' --columns "sa_id, sa_stu_id" --input-null-string 'null' --input-null-non-string 'null'

连接到 mysql 权限也没问题，账号可写，如果--table 参数 表名打错会提示没有这个表。

这个 tb_stu_answer_test 是我用 hql create table tb_stu_answer_test as select xx from another_table limit 2; 生成的子集。

错误信息如下(没啥信息量的 log )：

18/03/16 15:23:23 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

18/03/16 15:23:23 INFO tool.CodeGenTool: Beginning code generation

Fri Mar 16 15:23:24 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

18/03/16 15:23:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tb_stu_answer` AS t LIMIT 1

18/03/16 15:23:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tb_stu_answer` AS t LIMIT 1

18/03/16 15:23:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce

Note: /tmp/sqoop-/compile/8e6c405c3c24bd055881ab1a4ce27300/tb_stu_answer.java uses or overrides a deprecated API.

Note: Recompile with -Xlint:deprecation for details.

18/03/16 15:23:25 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-/compile/8e6c405c3c24bd055881ab1a4ce27300/tb_stu_answer.jar

18/03/16 15:23:25 INFO mapreduce.ExportJobBase: Beginning export of tb_stu_answer

18/03/16 15:23:26 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar

18/03/16 15:23:26 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative

18/03/16 15:23:26 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative

18/03/16 15:23:26 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

18/03/16 15:23:27 INFO client.RMProxy: Connecting to ResourceManager at aly-bigdata-hadoop-yarnmanager...com/10.255.25.38:8032

18/03/16 15:23:27 INFO hdfs.DFSClient: Created token for hive: HDFS_DELEGATION_TOKEN [email protected], renewer=yarn, realUser=, issueDate=1521185007407, maxDate=1521789807407, sequenceNumber=64697, masterKeyId=75 on 10.255.25.36:8020

18/03/16 15:23:27 INFO security.TokenCache: Got dt for hdfs://aly-bigdata-hadoop-namenodemaster...com:8020; Kind: HDFS_DELEGATION_TOKEN, Service: 10.255.25.36:8020, Ident: (token for hive: HDFS_DELEGATION_TOKEN [email protected], renewer=yarn, realUser=, issueDate=1521185007407, maxDate=1521789807407, sequenceNumber=64697, masterKeyId=75)

18/03/16 15:23:28 INFO input.FileInputFormat: Total input paths to process : 1

18/03/16 15:23:28 INFO input.FileInputFormat: Total input paths to process : 1

18/03/16 15:23:28 INFO mapreduce.JobSubmitter: number of splits:4

18/03/16 15:23:28 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1520934182986_5322

18/03/16 15:23:28 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: 10.255.25.36:8020, Ident: (token for hive: HDFS_DELEGATION_TOKEN [email protected], renewer=yarn, realUser=, issueDate=1521185007407, maxDate=1521789807407, sequenceNumber=64697, masterKeyId=75)

18/03/16 15:23:28 INFO impl.YarnClientImpl: Submitted application application_1520934182986_5322

18/03/16 15:23:28 INFO mapreduce.Job: The url to track the job: http://aly-bigdata-hadoop-yarnmanager...com:8088/proxy/application_1520934182986_5322/

18/03/16 15:23:28 INFO mapreduce.Job: Running job: job_1520934182986_5322

18/03/16 15:23:35 INFO mapreduce.Job: Job job_1520934182986_5322 running in uber mode : false

18/03/16 15:23:35 INFO mapreduce.Job: map 0% reduce 0%

18/03/16 15:23:40 INFO mapreduce.Job: map 100% reduce 0%

18/03/16 15:23:40 INFO mapreduce.Job: Job job_1520934182986_5322 failed with state FAILED due to: Task failed task_1520934182986_5322_m_000001

Job failed as tasks failed. failedMaps:1 failedReduces:0

18/03/16 15:23:40 INFO mapreduce.Job: Counters: 8

Job Counters

Failed map tasks=4

Launched map tasks=4

Rack-local map tasks=4

Total time spent by all maps in occupied slots (ms)=9805

Total time spent by all reduces in occupied slots (ms)=0

Total time spent by all map tasks (ms)=9805

Total vcore-milliseconds taken by all map tasks=9805

Total megabyte-milliseconds taken by all map tasks=10040320

18/03/16 15:23:40 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead

18/03/16 15:23:40 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 14.0035 seconds (0 bytes/sec)

18/03/16 15:23:40 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead

18/03/16 15:23:40 INFO mapreduce.ExportJobBase: Exported 0 records.

18/03/16 15:23:40 ERROR tool.ExportTool: Error during export:

Export job failed!

at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:439)

at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)

at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)

at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)

at org.apache.sqoop.Sqoop.run(Sqoop.java:147)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)

at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

有人遇到过类似问题有经验的么，文档看了没啥帮助，google，Stack Overflow 似乎遇到类似问题的人不多。