zoukankan      html  css  js  c++  java
  • ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    在学习spark时。spark运行会报一些无用的错误:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    文章发自:http://www.cnblogs.com/hark0623/p/4170172.html  转发请注明
    
    
    14/12/17 19:18:53 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
    java.io.IOException: Could not locate executable nullinwinutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
        at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
        at org.apache.hadoop.security.Groups.<init>(Groups.java:77)
        at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255)
        at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:283)
        at org.apache.spark.deploy.SparkHadoopUtil.<init>(SparkHadoopUtil.scala:36)
        at org.apache.spark.deploy.SparkHadoopUtil$.<init>(SparkHadoopUtil.scala:109)
        at org.apache.spark.deploy.SparkHadoopUtil$.<clinit>(SparkHadoopUtil.scala)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:228)
        at org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:555)
        at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:75)
        at com.hark.Test$.main(Test.scala:28)
        at com.hark.Test.main(Test.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
    14/12/17 19:18:54 INFO AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@WIN7-20140319ZQ:60477/user/HeartbeatReceiver

    windows的idea运行时报这个异常

    只要在代码前面添加

    //引入: import java.io.File
    
    val path = new File(".").getCanonicalPath()
    //File workaround = new File(".");
    System.getProperties().put("hadoop.home.dir", path);
    new File("./bin").mkdirs();
    new File("./bin/winutils.exe").createNewFile();

    ---这个是转发别人的: http://www.cnblogs.com/hark0623/p/4170172.html

     

  • 相关阅读:
    Android Widget小组件开发(一)——Android实现时钟Widget组件的步骤开发,这些知识也是不可缺少的!
    java基础——static keyword小节
    iOS UI08_UITableView
    Android 6.0 开发人员对系统权限的使用与练习(Permissions Best Practices)
    Android简单实现BroadCastReceiver广播机制
    UI组件之TextView及其子类(一)TextView和EditText
    BZOJ 3112 [Zjoi2013]防守战线 线性规划
    反射 + 配置文件 实现IOC容器
    android:异步任务asyncTask介绍及异步任务下载图片(带进度条)
    【LeetCode-面试算法经典-Java实现】【058-Length of Last Word (最后一个单词的长度)】
  • 原文地址:https://www.cnblogs.com/kaiwen1/p/8477445.html
Copyright © 2011-2022 走看看