zoukankan      html  css  js  c++  java
  • Wrong FS: hdfs://centos1:9000/user/hadoop/input, expected: file:///

    java读取hdfs时报错:

    Wrong FS: hdfs://centos1:9000/user/hadoop/input, expected: file:///

    首先确认 是用 FileSystem.get(conf) 而不是 FileSystem.getLocal(conf) 初始化的 FileSystem 对象.

    如果还有问题: 把集群里的 core-site.xml mapped-site.xml hdfs-site.xml slaves 等文件拷贝到eclipse 配置的 Hadoop 目录里.

    //hdfs://centos1:9000//user/hadoop/input hdfs://centos1:9000//user/hadoop/output
    	public static void main(String[] args) throws Exception {
    		Configuration conf = new Configuration();
    		Path inputPath = new Path("hdfs://centos1:9000/user/hadoop/input/");
    		//FileSystem file = FileSystem.getLocal(conf);
    		FileSystem file = FileSystem.get(conf);
    		FileStatus[] fileStatus = file.listStatus(inputPath);
    		byte[] temp = new byte[1024];
    		if(fileStatus!=null && fileStatus.length>0){
    			for (FileStatus status : fileStatus) {
    				System.out.println(status.getPath().getName() + "--"+ status.getLen()/1024/1024.0);
    				FSDataInputStream in = file.open(status.getPath());
    				int i = 0;
    				while((in.read(temp))!=-1){
    					System.out.println(new String(temp, 0, i));
    				}
    				in.close();
    			}
    		}
    		file.close();
    		System.out.println("end.");
    	}
    

      

  • 相关阅读:
    bzoj1081 [SCOI2005]超级格雷码
    bzoj3790 神奇项链
    bzoj2822 [AHOI2012]树屋阶梯
    bzoj1485 [HNOI2009]有趣的数列
    bzoj1486 [HNOI2009]最小圈
    bzoj2721 [Violet 5]樱花
    POJ 1238 Substrings
    ZOJ Team Formation
    POJ 1459 Power Network
    POJ 1458 Common Subsequence
  • 原文地址:https://www.cnblogs.com/BennyTian/p/3005773.html
Copyright © 2011-2022 走看看