zoukankan      html  css  js  c++  java
  • HDFS不存在绝对路径,无法找到文件所在具体位置

    This is set in the dfs.datanode.data.dir property, which defaults to file://${hadoop.tmp.dir}/dfs/data (see details here).

    However, in your case, the problem is that you are not using the full path within HDFS. Instead, do:

    hadoop fs -ls /usr/local/myhadoop-tmp/

    Note that, you also seem to be confusing the path within HDFS to the path in your local file system. Within HDFS, your file is in /usr/local/myhadoop-tmp/. In your local system (and given your configuration setting), it is under /usr/local/myhadoop-tmp/dfs/data/; in there, there's a directory structure and naming convention defined by HDFS, that is independent to whatever path in HDFS you decide to use. Also, it won't have the same name, since it is divided into blocks and each block is assigned a unique ID; the name of a block is something like blk_1073741826.

    To conclude: the local path used by the datanode is NOT the same as the paths you use in HDFS. You can go into your local directory looking for files, but you should not do this, since you could mess up the HDFS metadata management. Just use the hadoop command-line tools to copy/move/read files within HDFS, using any logical path (in HDFS) that you wish to use. These paths within HDFS do not need to be tied to the paths you used in for your local datanode storage (there is no reason to or advantage of doing this).

  • 相关阅读:
    PHP+NGINX
    mysql添加索引
    SecureFX 乱码问题
    kali ssh服务连接问题,无法远程管理
    Nginx安装配置
    redhat mysql安装配置
    shell脚本-1
    Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks-paper
    Parsing with Compositional Vector Grammars--paper
    Reasoning With Neural Tensor Networks For Knowledge Base Completion-paper
  • 原文地址:https://www.cnblogs.com/zwingblog/p/6800624.html
Copyright © 2011-2022 走看看