zoukankan      html  css  js  c++  java
  • Spark-shell 启动WARN---java.net.BindException: Address already in use

    同时打开了两个SecureCRT的终端界面,其中一个已经进入了Spark-shell,在另一个SecureCRT界面中执行

    "spark-shell --master yarn --deploy-mode client < cha02.scala" 时报错,如下:

    [root@master test_code]# spark-shell --master yarn --deploy-mode client <cha02.scala
    Setting default log level to "WARN".
    To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
    17/05/07 20:41:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    17/05/07 20:41:37 WARN component.AbstractLifeCycle: FAILED ServerConnector@1953bc95{HTTP/1.1}{0.0.0.0:4040}: java.net.BindException: Address already in use
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:433)
    at sun.nio.ch.Net.bind(Net.java:425)

    17/05/07 21:00:46 WARN util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
    17/05/07 21:00:59 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
    17/05/07 21:01:37 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
    Spark context Web UI available at http://192.168.1.200:4041
    Spark context available as 'sc' (master = yarn, app id = application_1494142860645_0005).
    Spark session available as 'spark'.
    Welcome to
    ____ __
    / __/__ ___ _____/ /__
    _ / _ / _ `/ __/ '_/
    /___/ .__/\_,_/_/ /_/\_ version 2.1.0
    /_/

    Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_121)
    Type in expressions to have them evaluated.
    Type :help for more information.

    scala>

    这是由于启动一个Spark context 时,SparkUI 默认会使用 4040 端口,当 4040 端口被占用时,则尝试使用另外一个端口,即日志中的 

    Service 'SparkUI' could not bind on port 4040. Attempting port 4041.

  • 相关阅读:
    基于Antlr4编写DSL
    【整理】ANTLR应用案例 | 在路上
    【整理】ANTLR应用案例 | 在路上
    The ANTLR Parser Generator
    ANTLR4权威参考手册
    ANTLR Examples
    ANTLRWorks: The ANTLR GUI Development Environment
    http://www.cnblogs.com/vowei/archive/2012/08/24/2654287.html
    写一个编译器
    写一个编译器
  • 原文地址:https://www.cnblogs.com/followyourdream/p/6822308.html
Copyright © 2011-2022 走看看