zoukankan      html  css  js  c++  java
  • HDFS不存在绝对路径,无法找到文件所在具体位置

    This is set in the dfs.datanode.data.dir property, which defaults to file://${hadoop.tmp.dir}/dfs/data (see details here).

    However, in your case, the problem is that you are not using the full path within HDFS. Instead, do:

    hadoop fs -ls /usr/local/myhadoop-tmp/

    Note that, you also seem to be confusing the path within HDFS to the path in your local file system. Within HDFS, your file is in /usr/local/myhadoop-tmp/. In your local system (and given your configuration setting), it is under /usr/local/myhadoop-tmp/dfs/data/; in there, there's a directory structure and naming convention defined by HDFS, that is independent to whatever path in HDFS you decide to use. Also, it won't have the same name, since it is divided into blocks and each block is assigned a unique ID; the name of a block is something like blk_1073741826.

    To conclude: the local path used by the datanode is NOT the same as the paths you use in HDFS. You can go into your local directory looking for files, but you should not do this, since you could mess up the HDFS metadata management. Just use the hadoop command-line tools to copy/move/read files within HDFS, using any logical path (in HDFS) that you wish to use. These paths within HDFS do not need to be tied to the paths you used in for your local datanode storage (there is no reason to or advantage of doing this).

  • 相关阅读:
    Elasticsearch 机制 架构 集群 选举
    《Leo the late bloomer》阿虎开窍了
    各行业发明专利排行榜
    接口响应 越来越慢
    知识产权代理 与 工作流
    MBA Business Org Responsbility Account Stock
    心理学 防内耗
    Apache Kafka Zookeeper Quorum
    The different aspect of architecture(架构的不同方面)
    hPaPaas low-code/no-code 低代码
  • 原文地址:https://www.cnblogs.com/zwingblog/p/6800624.html
Copyright © 2011-2022 走看看