zoukankan      html  css  js  c++  java
  • spark常见问题定位

    1.eclipse下报java.io.IOException: Could not locate executable nullinwinutils.exe in the Hadoop binaries.
    在https://github.com/srccodes/hadoop-common-2.2.0-bin页面下载hadoop-common-2.2.0-bin-master.zip,解压,设置HADOOP_HOME环境变量为解压后的地址
    参考资料:http://my.oschina.net/cloudcoder/blog/286234
    2.eclipse下报ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up.

    查看spark的log目录下的spark-hadoop-org.apache.spark.deploy.master.Master-1-h1.out
    ./bin/spark-submit --class org.apache.spark.examples.SparkPi  --master spark://h1:7077  --executor-memory 3000m  --total-executor-cores 100  ./lib/spark-examples-1.4.0-hadoop2.6.0.jar 1000
    将命令的ip换成host
    3.spark java.lang.StackOverflowError
    a.检查下有没有死循环
    b.深层次的递归
    c.设定jvm参数:-Xss10m
    4.WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

    ./bin/run-example SparkPi --spark.executor.memory 3000m
    5.Cannot change version of project facet Dynamic Web Module to 2.5.
    http://blog.csdn.net/steveguoshao/article/details/38414145

    6.Exception in thread "main" java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET

    有两个版本的http-core,从pom中去掉一个

  • 相关阅读:
    indy 發郵件的一些說明
    一此常用三方组件的地址(持续更新)
    oraSession直连
    获取关机信息及键盘或鼠标无响应的时长
    DBConnection释放说明
    发送邮件
    文件操作
    plsql中的一些知識
    TNativeXML用法(轉)
    redux中createStore, conbineReducers的简易封装
  • 原文地址:https://www.cnblogs.com/sunny08/p/4882486.html
Copyright © 2011-2022 走看看