mysql hadoop sqoop,使用sqoop（1.4.5）将mysql数据导入到hdfs（hadoop 2.4.1）

I am a begineer in Hadoop, trying to import a table from my local database mysql, to hdfs of hadoop 2.4.1 using sqoop 1.4.5.

mysql version : 5.5.38

hdfs version :hadoop 2.4.1

sqoop version :1.4.5

i try sqoop-list-databases ,it give correct answer

but then on giving following cmd:

sqoop import --connect jdbc:mysql://localhost:3306/information_schema --username root --password root --table VIEWs -m 1

and i get:

ERROR tool.ImportTool: Encountered IOException running import job: java.io.FileNotFoundException: File does not exist: hdfs://localhost:54310/usr/lib/sqoop/lib/jackson-mapper-asl-1.9.13.jar

at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)

at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)

at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)

at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)

at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)

at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)

at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)

at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)

at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)

at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)

at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)

at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:247)

at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:665)

at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)

at org.apache.sqoop.Sqoop.run(Sqoop.java:143)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)

at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

解决方案

Your mapred-site.xml file is configured wrongly. Check the file content, The content should looks like shown below:

mapreduce.framework.name

yarn

Generally mapred-site.xml file content is made the same as core-site.xml content. Core-site.xml file content should looks like shown below:

fs.default.name

hdfs://master:54310

Tayfun YALCINKAYA