zoukankan      html  css  js  c++  java
  • 大数据学习——上传本地文件到集群根目录下

    TestHDFS.java

    package cn.itcast.hdfs;
    
    import java.io.IOException;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.FileSystem;
    import org.apache.hadoop.fs.Path;
    
    public class TestHDFS {
        public static void main(String[] args) throws IOException {
            Configuration conf = new Configuration();
            //1首先需要一个hdfs的客户端对象
            conf.set("fs.defaultFS", "hdfs://mini1:9000");
            FileSystem fs = FileSystem.get(conf);
            fs.copyFromLocalFile(new Path("E://he.txt"), new Path("/"));
            fs.close();
        }
    }

    此时报错,权限不足:

    修改后的代码(伪造一个root身份):

    package cn.itcast.hdfs;
    
    import java.io.IOException;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.FileSystem;
    import org.apache.hadoop.fs.Path;
    
    public class TestHDFS {
        public static void main(String[] args) throws IOException {
            Configuration conf = new Configuration();
    
            //伪造root用户身份
            System.setProperty("HADOOP_USER_NAME","root");
    
            //1首先需要一个hdfs的客户端对象
            conf.set("fs.defaultFS", "hdfs://mini1:9000");
            FileSystem fs = FileSystem.get(conf);
            fs.copyFromLocalFile(new Path("E://he.txt"), new Path("/"));
            fs.close();
        }
    }

    或者是运行时加一个参数:

    VM-arguements:

    -DHADOOP——USER_NAME=root

  • 相关阅读:
    Codeforces Round #375 (Div. 2)
    ACM之路(19)—— 主席树初探
    CodeForces 722D Generating Sets
    CodeForces 721D Maxim and Array
    心情--2014区域赛
    【N-Quens II】cpp
    【N-Queens】cpp
    【Unique Paths II】cpp
    【Unique Paths】cpp
    【Palindrome Partitioning】cpp
  • 原文地址:https://www.cnblogs.com/feifeicui/p/10216317.html
Copyright © 2011-2022 走看看