sqoop mysql date_Sqoop_ 从 hive 导到mysql常遇九问题总结(转)

以前以为版本不同，遇到的问题就不同，后来发现，无论是新版本，还是老版本，遇到的问题大部分都是相同的。下面解决问题的方法仅供借鉴

1.拒绝连接的错误表现是什么？

2.表不存在该如何解决？

3.null字段填充符该如何指定？

环境hive 版本hive-0.11.0

sqoop 版本 sqoop-1.4.4.bin__hadoop-1.0.0

从hive导到mysql

mysql 表

mysql> desc cps_activation;

+————+————-+——+—–+———+—————-+

| Field | Type | Null | Key | Default | Extra |

+————+————-+——+—–+———+—————-+

| id | int(11) | NO | PRI | NULL | auto_increment |

| day | date | NO | MUL | NULL | |

| pkgname | varchar(50) | YES | | NULL | |

| cid | varchar(50) | YES | | NULL | |

| pid | varchar(50) | YES | | NULL | |

| activation | int(11) | YES | | NULL | |

+————+————-+——+—–+———+—————-+

6 rows in set (0.01 sec)

hive表

hive> desc active;

OK

id int None

day string None

pkgname string None

cid string None

pid string None

activation int None

测试链接成功

[hadoop@hs11 ~]sqoop list-databases –connect jdbc:mysql://localhost:3306/ –username root –password admin

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:42:26 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:42:26 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

information_schema

easyhadoop

mysql

test

[hadoop@hs11 ~]$ sqoop list-databases –connect jdbc:mysql://localhost:3306/test –username root –password admin

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:42:40 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:42:40 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

information_schema

easyhadoop

mysql

test

[hadoop@hs11 ~]$ sqoop list-tables –connect jdbc:mysql://localhost:3306/test –username root –password admin

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:42:54 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:42:54 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

active

[hadoop@hs11 ~]$ sqoop create-hive-table –connect jdbc:mysql://localhost:3306/test –table active –username root –password admin –hive-table test

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:57:04 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:57:04 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override

13/08/20 16:57:04 INFO tool.BaseSqoopTool: delimiters with –fields-terminated-by, etc.

13/08/20 16:57:04 WARN tool.BaseSqoopTool: It seems that you’ve specified at least one of following:

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –hive-home

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –hive-overwrite

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –create-hive-table

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –hive-table

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –hive-partition-key

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –hive-partition-value

13/08/20 16:57:04 WARN tool.BaseSqoopTool: –map-column-hive

13/08/20 16:57:04 WARN tool.BaseSqoopTool: Without specifying parameter –hive-import. Please note that

13/08/20 16:57:04 WARN tool.BaseSqoopTool: those arguments will not be used in this session. Either

13/08/20 16:57:04 WARN tool.BaseSqoopTool: specify –hive-import to apply them correctly or remove them

13/08/20 16:57:04 WARN tool.BaseSqoopTool: from command line to remove this warning.

13/08/20 16:57:04 INFO tool.BaseSqoopTool: Please note that –hive-home, –hive-partition-key,

13/08/20 16:57:04 INFO tool.BaseSqoopTool: hive-partition-value and –map-column-hive options are

13/08/20 16:57:04 INFO tool.BaseSqoopTool: are also valid for HCatalog imports and exports

13/08/20 16:57:04 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

13/08/20 16:57:05 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `active` AS t LIMIT 1

13/08/20 16:57:05 INFO