zoukankan      html  css  js  c++  java
  • Hadoop创建/删除文件夹出错

    log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
    log4j:WARN Please initialize the log4j system properly.
    log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
    Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://sandbox.hortonworks.com:8020/user/testdir, expected: file:///
        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
        at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
        at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:590)
        at com.hdfs.directory.CreateDir.main(CreateDir.java:19)
    在本机运行是会出现上面的错误,改成下面的代码:

    package com.hdfs.directory;
    
    import java.io.IOException;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.FileSystem;
    import org.apache.hadoop.fs.Path;
    
    public class CreateDir {
    
        /**
         * @param args
         * @throws IOException 
         */
        public static void main(String[] args) throws IOException {
            // TODO Auto-generated method stub
            Configuration conf=new Configuration();
            Path newPath = new Path("hdfs://sandbox.hortonworks.com:8020/user/testdir");
            //FileSystem fs=FileSystem.get(conf);
            FileSystem fs = newPath.getFileSystem(conf);
            fs.mkdirs(newPath);
            fs.close();
    //        FileStatus fileStatus=fs.getFileStatus(new Path("/user/long1657/20130908"));
            
        }
    
    }

    image

    或者hadoop相关的配置信息,放在工程目录下:

     

    image

    执行上面的代码后:/user/testdir目录被成功删除:

    image

     

    在将hadoop相关的配置文件添加到工程中后,用下面的代码就可以成功创建文件夹:

    package com.hdfs.directory;
    
    import java.io.IOException;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.FileSystem;
    import org.apache.hadoop.fs.Path;
    
    public class CreateDirWithConf {
    
        /**
         * @param args
         * @throws IOException 
         */
        public static void main(String[] args) throws IOException {
            // TODO Auto-generated method stub
            Configuration conf=new Configuration();
        
            FileSystem fs=FileSystem.get(conf);
    
            fs.mkdirs(new Path("hdfs://sandbox.hortonworks.com:8020/user/testdir"));
            fs.close();
    //        FileStatus fileStatus=fs.getFileStatus(new Path("/user/long1657/20130908"));
            
        }
    
    }

     

  • 相关阅读:
    [单选题]请求文件“time.inc”,当发生错误时就终止脚本,正确的方式是:
    [单选题]条件语句的时候不应该使用哪一种控制结构
    [高德地图]学习笔记--基本结构
    nodejs实战:小爬虫
    linux实用命令(2016/11/8-永远)
    自适应响应式布局-实现原理
    解决npm安装慢的方法
    git进阶(分支与标签管理)
    git进阶(远程仓库github)
    git入门命令(只涉及本地仓库管理)
  • 原文地址:https://www.cnblogs.com/justinzhang/p/4844137.html
Copyright © 2011-2022 走看看