【报错】sqoop执行 list-tables 和 list_databases都OK,但是import有问题Communications link failure

我再云服务器上搭了hadoop的伪分布式环境,在使用sqoop将mysql中的表数据导入到hdfs时报错,而使用sqoop的st-tables 和 list_databases都是没有问题的,能正常的查询到数据,偏偏import不可以,并且在使用import的时候,连接的mysql服务直接挂了~~哪位大神指教指教

这个问题解决了,分享一下经验:

前言:每个人遇到的问题原因可能不一样,这里只说我遇到的问题和解决方法。
先说解决办法:卸载mysql重新装!!!!(下下策)
场景:每次使用sqoop的st-tables 和 list_databases都是没有问题的,能正常的查询到数据,偏偏import不可以,并且在使用import的时候,MySQL的服务直接报错关闭了,再次尝试启动mysql的服务时也启动不了了,找了搞运维的同学看了下,发现/etc/mysql/my.cnf文件有异常,简单的办法就是直接删掉,试试再试试,如果不行就直接重装,现在具体为什么不行我也还没找到原因,不过可以肯定的是,这个坑是在安装mysql的时候挖下的,实在找不到办法了重装吧。。。如果哪位小伙伴找到了原因,欢迎留言,拜谢。

报错信息:


[root@hwyuntrx hadoop]# sqoop import --connect jdbc:mysql://127.0.0.1:3306/test --username root --password xiaojiajia --table userinfo --columns ‘id, name’
Warning: /usr/sqoop/…/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/sqoop/…/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/sqoop/…/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/sqoop/…/zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/03/06 17:50:28 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/03/06 17:50:28 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/03/06 17:50:29 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/03/06 17:50:29 INFO tool.CodeGenTool: Beginning code generation
19/03/06 17:50:29 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM userinfo AS t LIMIT 1
19/03/06 17:50:29 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM userinfo AS t LIMIT 1
19/03/06 17:50:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hadoop/hadoop2.8
Note: /tmp/sqoop-root/compile/dcc1312d91efeac355eea003e1a7ea70/userinfo.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/03/06 17:50:31 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/dcc1312d91efeac355eea003e1a7ea70/userinfo.jar
19/03/06 17:50:31 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/03/06 17:50:31 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/03/06 17:50:31 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/03/06 17:50:31 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/03/06 17:50:31 INFO mapreduce.ImportJobBase: Beginning import of userinfo
19/03/06 17:50:31 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/03/06 17:50:31 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/03/06 17:50:32 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/03/06 17:50:33 INFO client.RMProxy: Connecting to ResourceManager at hwyuntrx/172.16.0.118:8032
19/03/06 17:50:35 INFO db.DBInputFormat: Using read commited transaction isolation
19/03/06 17:50:35 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(id), MAX(id) FROM userinfo
19/03/06 17:50:35 INFO db.IntegerSplitter: Split size: 1; Num splits: 4 from: 1 to: 5
19/03/06 17:50:35 INFO mapreduce.JobSubmitter: number of splits:5
19/03/06 17:50:35 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1551854254469_0008
19/03/06 17:50:36 INFO impl.YarnClientImpl: Submitted application application_1551854254469_0008
19/03/06 17:50:36 INFO mapreduce.Job: The url to track the job: http://hwyuntrx:8088/proxy/application_1551854254469_0008/
19/03/06 17:50:36 INFO mapreduce.Job: Running job: job_1551854254469_0008
19/03/06 17:50:49 INFO mapreduce.Job: Job job_1551854254469_0008 running in uber mode : false
19/03/06 17:50:49 INFO mapreduce.Job: map 0% reduce 0%
19/03/06 17:51:09 INFO mapreduce.Job: Task Id : attempt_1551854254469_0008_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)
… 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
at com.mysql.jdbc.MysqlIO.(MysqlIO.java:350)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2408)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2445)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2230)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:813)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:399)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:334)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:216)
… 11 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.(Socket.java:434)
at java.net.Socket.(Socket.java:244)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259)
at com.mysql.jdbc.MysqlIO.(MysqlIO.java:300)
… 27 more

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值