zoukankan      html  css  js  c++  java
  • troubshooting-sqoop 导出 TiDB表数据报com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    2019-11-22 17:15:27,705 FATAL [IPC Server handler 13 on 44844] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1574410493054_0011_m_000000_0 - exited : java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

    at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)

    at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)

    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)

    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)

    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:755)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)

    at java.security.AccessController.doPrivileged(Native Method)

    at javax.security.auth.Subject.doAs(Subject.java:422)

    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)

    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

    Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

    at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)

    at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)

    ... 10 more

    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    原因分析

    1、TiDB数据库连接驱动包(当前是用的 mysql数据库连接驱动包)版本与 CDH集群的 sqoop组件版本不兼容。

    2、CDH集群中的单一节点可以访问 TiDB数据库,其它节点与 TiDB数据库主机网络不通(假如 CDH集群有5个节点,需要5个节点都可以与 mysql的主机网络互通)。

    此次报错原因是第二点。

    解决办法

    1、更新兼容当前 TiDB数据库连接驱动包(当前是用的 mysql数据库连接驱动包)与 CDH集群的 sqoop组件版本的驱动包。

    2、关闭 CDH集群节点访问 TiDB数据库主机的防火墙,使得 CDH集群主机任何节点都可以访问 TiDB数据库。

    异常日志

    2019-11-22 17:15:27,705 FATAL [IPC Server handler 13 on 44844] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1574410493054_0011_m_000000_0 - exited : java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

    at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)

    at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)

    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)

    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)

    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:755)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)

    at java.security.AccessController.doPrivileged(Native Method)

    at javax.security.auth.Subject.doAs(Subject.java:422)

    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)

    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

    Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

    at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)

    at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)

    ... 10 more

    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

    The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)

    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)

    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:348)

    at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2391)

    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2428)

    at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2213)

    at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:797)

    at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)

    at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)

    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)

    at java.sql.DriverManager.getConnection(DriverManager.java:664)

    at java.sql.DriverManager.getConnection(DriverManager.java:247)

    at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)

    at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:216)

    ... 11 more

    Caused by: java.net.ConnectException: Connection timed out (Connection timed out)

    at java.net.PlainSocketImpl.socketConnect(Native Method)

    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

    at java.net.Socket.connect(Socket.java:589)

    at java.net.Socket.connect(Socket.java:538)

    at java.net.Socket.<init>(Socket.java:434)

    at java.net.Socket.<init>(Socket.java:244)

    at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:257)

    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:298)

    ... 27 more

  • 相关阅读:
    手写spring事务框架, 揭秘AOP实现原理。
    centos7修改端口登陆
    数据库的锁机制
    linux安装mysql5.6
    SpringMVC数据格式化
    Java处理小数点后几位
    docker学习(七)常见仓库介绍
    docker学习(六) Docker命令查询
    docker学习(六)
    docker学习(五)
  • 原文地址:https://www.cnblogs.com/chwilliam85/p/11918655.html
Copyright © 2011-2022 走看看