zoukankan      html  css  js  c++  java
  • 解决spark程序报错:Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]

    报错信息:

    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:49)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.aggregate.TungstenAggregate.doExecute(TungstenAggregate.scala:80)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Exchange.prepareShuffleDependency(Exchange.scala:164)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Exchange$$anonfun$doExecute$1.apply(Exchange.scala:254)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Exchange$$anonfun$doExecute$1.apply(Exchange.scala:248)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:48)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     ... 64 more
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO - Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at scala.concurrent.Await$.result(package.scala:107)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.joins.BroadcastHashJoin.doExecute(BroadcastHashJoin.scala:107)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.Project.doExecute(basicOperators.scala:46)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1.apply(TungstenAggregate.scala:86)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1.apply(TungstenAggregate.scala:80)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:48)
    09-05-2017 09:58:44 CST xxxx_job_1494294485570174 INFO -     ... 73 more
    根据最后的Caused by信息和stack trace信息进行搜索,确定是broacast阶段超时,解决方法:
  • 相关阅读:
    WorkFlow入门Step.2—Building a Simple WorkFlowForWF4.0
    AgileEAS.NET敏捷开发平台升级版(丑小鸭的蜕变)[已修复下载链接]
    AgileEAS.NET平台开发实例药店系统视频教程系列索引
    AgileEAS.NET敏捷开发平台及案例下载(持续更新)索引
    WorkFlow入门Step.3—Adding Procedural ElementsForWF4.0
    AgileEAS.NET敏捷开发平台案例药店系统项目综述
    AgileEAS.NET平台开发实例智能升级及服务器部署
    WorkFlow入门Step.4—Adding Procedural ElementsForWF4.0(续)
    AgileEAS.NET平台开发实例药店系统资源文件的替换[大家关心的问题]
    停靠在八楼的2路汽车/ 刀郎:2002年第一场雪
  • 原文地址:https://www.cnblogs.com/aprilrain/p/6916311.html
Copyright © 2011-2022 走看看