zoukankan      html  css  js  c++  java
  • HDFS文件操作

    文件操作

    Hadoop-fs 帮助文档

    [root@hadoop01 current]# hadoop fs –help

    -ls [-d] [-h] [-R] [<path> ...] :

      List the contents that match the specified file pattern. If path is not

      specified, the contents of /user/<currentUser> will be listed. Directory entries

      are of the form:

            permissions - userId groupId sizeOfDirectory(in bytes)

      modificationDate(yyyy-MM-dd HH:mm) directoryName

      

      and file entries are of the form:

            permissions numberOfReplicas userId groupId sizeOfFile(in bytes)

      modificationDate(yyyy-MM-dd HH:mm) fileName

                                                                                     

      -d  Directories are listed as plain files.                                     

      -h  Formats the sizes of files in a human-readable fashion rather than a number

          of bytes.                                                                  

      -R  Recursively list the contents of directories.

    查看所有文件

    hadoop fs -ls /

    下载文件

    hadoop fs -get hdfs://hadoop01:9000/jdk* /home/software/

    hdfs根目录下载以jdk开头的文件,下载到/home/software目录

    copyFromLocal

    从本地copy一个文件到hdfs系统中

    [root@hadoop01 software]# hadoop fs -copyFromLocal /home/software/jdk-7u55-linux-i586.tar.gz  /

    说明:源路径只能是本地的一个文件

    put

    Hadoop fs –put /home/software/a.txt /input

    /home/software下面的a.txt文件上传到hdfs的input文件夹

    文件个数

    hadoop fs -count /

    统计文件大小

    这种情况下计算出来的大小就一个数据

    [root@hadoop01 software]# hadoop fs -du -s -h hdfs://hadoop01:9000/

    Java HotSpot(TM) Client VM warning: You have loaded library /home/app/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.

    It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

    15/03/24 01:07:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    133.0 M  hdfs://hadoop01:9000/

    这种情况下计算出来的大小是每一个文件的

    [root@hadoop01 software]# hadoop fs -du -s -h hdfs://hadoop01:9000/*

    Java HotSpot(TM) Client VM warning: You have loaded library /home/app/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.

    It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

    15/03/24 01:07:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    133.0 M  hdfs://hadoop01:9000/jdk-7u55-linux-i586.tar.gz

    9  hdfs://hadoop01:9000/slaves

    删除文件

    [root@hadoop08 ~]# hadoop fs -rm -r /output

  • 相关阅读:
    中国历史朝代公元对照简表
    [Solved] DashBoard – Excel Service: The data sources may be unreachable, may not be responding, or may have denied you access.
    Delete/Remove Project from TFS 2010
    Sharepoint site showing system account instead of my username on the top right corner.
    你的成功在于你每天养成的习惯
    Internet Information Services is running in 32bit emulation mode. Correct the issue listed above and rerun setup.
    Prepare to back up and restore a farm (Office SharePoint Server 2007)
    Word中字号与磅值的对应关系
    How to: Change the Frequency for Refreshing the Data Warehouse for Team System
    UI Automation in WPF/Silverlight
  • 原文地址:https://www.cnblogs.com/zpb2016/p/5788412.html
Copyright © 2011-2022 走看看