zoukankan      html  css  js  c++  java
  • 解决spark程序报错:Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]

    报错信息:

    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:49)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.aggregate.TungstenAggregate.doExecute(TungstenAggregate.scala:80)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Exchange.prepareShuffleDependency(Exchange.scala:164)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Exchange$$anonfun$doExecute$1.apply(Exchange.scala:254)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Exchange$$anonfun$doExecute$1.apply(Exchange.scala:248)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:48)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     ... 64 more
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.Await$.result(package.scala:107)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.joins.BroadcastHashJoin.doExecute(BroadcastHashJoin.scala:107)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Project.doExecute(basicOperators.scala:46)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1.apply(TungstenAggregate.scala:86)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1.apply(TungstenAggregate.scala:80)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:48)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     ... 73 more
    根据最后的Caused by信息和stack trace信息进行搜索,确定是broacast阶段超时,解决方法:
  • 相关阅读:
    今天下午去了佛山梁园——广东四大名园之一
    我眼中的Web2.0
    《花眼》观后感
    Implement SOAP services with the Zend Framework
    js 捕捉右键事件
    Zend Framework 第九节数据库操作学习总结
    PHP :time(),date(),mktime()日期与时间函数库{经常会忘却掉}
    zend_soap 实现 web service 用户验证
    php中DOMDocument简单用法(XML创建、添加、删除、修改)
    jquery 判断浏览器方法
  • 原文地址:https://www.cnblogs.com/aprilrain/p/6916311.html
Copyright © 2011-2022 走看看