sqoop: mysql导入数据到hdfs, hive, hbase

1. mysql 导入 hdfs

#启动hadoop
$ start-dfs.sh
$ start-yarn.sh
$ jps
2706 NameNode
3334 ResourceManager
3495 NodeManager
3112 SecondaryNameNode
3848 Jps
2873 DataNode

# sqoop  energydata是数据库名，average_price_by_state是表名
$ sqoop import --connect jdbc:mysql//localhost:3306/energydata --username abc -P --table average_price_by_state -m 1

检验结果

$ hadoop fs -tail average_price_by_state/part-m-00000 | less

2. mysql导入hive

先删除刚刚在Hadoop上的文件

$ dfs -rm -r /user/hadoop/average_price_by_state
$ sqoop import --connect jdbc:mysql://localhost:3306/energydata --username abc --table average_price_by_state -P --hive-import --fields-terminated-by ',' --lines-terminated-by '\n' --null-string 'null' -m 1

如果遇到报错：

 ERROR tool.ImportTool: Import failed: java.io.IOException: Exception thrown in Hive
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:358)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:4