zoukankan      html  css  js  c++  java
  • 【原创】大叔经验分享(13)spark运行报错WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.

    本地运行spark报错

    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
    18/12/18 12:56:55 ERROR SparkContext: Error initializing SparkContext.
    java.net.BindException: Cannot assign requested address: bind: Service 'sparkDriver' failed after 16 retries (starting from 0)! Consider explicitly setting the appropriate port for the service 'sparkDriver' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:433)
    at sun.nio.ch.Net.bind(Net.java:425)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
    at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:127)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:501)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
    at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
    at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
    at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
    at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:210)
    at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:353)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:455)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
    at java.lang.Thread.run(Thread.java:745)
    18/12/18 12:56:55 INFO SparkContext: Successfully stopped SparkContext

    看起来一直在尝试绑定端口0失败,默认起始端口应该是4040,设置spark.driver.port=4040还是报相同的错,后来发现增加环境变量即可

    SPARK_LOCAL_IP=127.0.0.1

  • 相关阅读:
    YUV图片旋转
    mac http&git代理配置
    iOS 如何保持线程一直在运转
    iOS 后台录音Tweak实现参考--stackoverflow
    NSTimer的精确度
    svn递归添加目录下面所有文件
    利用instruments工具查看其它app的性能
    iOS符号表手工还原
    Hbase的安装测试工作
    Hadoop集群中pig工具的安装过程记录
  • 原文地址:https://www.cnblogs.com/barneywill/p/10140026.html
Copyright © 2011-2022 走看看