zoukankan      html  css  js  c++  java
  • Hadoop Failed to set permissions of path

    在Eclipse里面写了个測试程序:把HDFS中的数据批量导入到HBase中


    写好后,在本地測试遇到了例如以下问题:

    14/04/21 16:49:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    14/04/21 16:49:53 ERROR security.UserGroupInformation: PriviledgedActionException as:admin cause:java.io.IOException: Failed to set permissions of path: mphadoop-adminmapredstagingadmin-1518427785.staging to 0700
    Exception in thread "main" java.io.IOException: Failed to set permissions of path: mphadoop-adminmapredstagingadmin-1518427785.staging to 0700
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:690)
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
    at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:918)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
    at com.plocc.hadoop.hbase.BatchImport.main(BatchImport.java:104)


    此时我把程序打包到Linux上执行是okey的,那说明是Windows下文件权限问题,

    然后依据at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:690)这个错误信息找到了源代码,

    然后一步步调试到Win32FileSystem.class中的

    public native boolean setPermission(File f, int access, boolean enable, boolean owneronly);方法,

    这个native格式的方法可能在c或者c++里面调用了,没法继续跟踪了,所以我这里的解决方法是把

    private static void checkReturnValue(boolean rv, File p, 
                                           FsPermission permission
                                           ) throws IOException {
           
     if (!rv) {
          throw new IOException("Failed to set permissions of path: " + p + 
                                " to " + 
                                String.format("%04o", permission.toShort()));
        }

      }
      

    标记经色部分的代码先凝视掉,保证在windows下測试通过。



  • 相关阅读:
    内存溢出OutOfMemory
    系统问题定位
    final域的内存语义
    volatile内存语义
    锁的内存语义
    happens-before
    python 安装brew(国内的自动安装)
    【Mac】点击dmg文件后无反应,解决方案
    Mac----xmind 8 下载与破解方法
    面试中可向HR 提的问题总结?
  • 原文地址:https://www.cnblogs.com/blfshiye/p/4046601.html
Copyright © 2011-2022 走看看