zoukankan      html  css  js  c++  java
  • spark streaming 异常No output streams registered, so nothing to execute

    实现spark streaming demo时,代码:

     1 public static void main (String[] args) {
     2 
     3 
     4 SparkConf conf = new SparkConf().setAppName("Spark_Streaming").setMaster("local");
     5         JavaSparkContext sc = new JavaSparkContext(conf);       
     6         JavaStreamingContext jssc = new JavaStreamingContext(sc, new Duration(2));      
     7         JavaSQLContext sqlCtx = new JavaSQLContext(sc);     
     8 
     9 
    10         String[] filters = new String[] {"soc"};
    11 
    12         JavaReceiverInputDStream<Status> receiverStream = TwitterUtils.createStream(jssc,filters);
    13 
    14 
    15 
    16          jssc.start();
    17          jssc.awaitTermination();
    18 
    19 }

    运行以上代码会出现异常:No output streams registered, so nothing to execute

    原因是当发生输出操作的时候,会触发流计算操作,当在 DStream上没有输出操作或者计算时,通常需要执行某些操作,比如以任意方法:

    1 print()
    2 foreachRDD(func)
    3 saveAsObjectFiles(prefix, [suffix])
    4 saveAsTextFiles(prefix, [suffix])
    5 saveAsHadoopFiles(prefix, [suffix])

     可参考:http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations

  • 相关阅读:
    var、let、const
    面向女朋友自我介绍
    ES6——class
    一个错误引发的——对异步回调与for循环(小白错误,大神勿进)
    关于this
    关于作用域
    HTML5 8
    HTML5 7
    HTML5 6
    HTML5 4
  • 原文地址:https://www.cnblogs.com/jessezeng/p/5177935.html
Copyright © 2011-2022 走看看