zoukankan      html  css  js  c++  java
  • hadoop hdfs 命令行 设置文件夹大小的上限 quota:配额

    》bin/hdfs dfs -put readme.txt /finance
    
    》bin/hdfs dfs -du -s /finance
    》3901  /finance
    
    》bin/hdfs dfsadmin -setSpaceQuota 4000 /finance
      设置/finance的大小上限为4000个字节
    
    》bin/hdfs dfs -put readme1.txt /finance
      由于已经超出4000个字节,则会报错
    
    14/03/04 11:56:31 WARN hdfs.DFSClient: DataStreamer Exception
    org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /finance is exceeded: quota = 4000【配额】 B = 3.91 KB but diskspace consumed = 268443258【消耗】 B = 256.01 MB
            at org.apache.hadoop.hdfs.server.namenode.INodeDirectoryWithQuota.verifyQuota(INodeDirectoryWithQuota.java:191)
            at org.apache.hadoop.hdfs.server.namenode.FSDirectory.verifyQuota(FSDirectory.java:2054)
            at org.apache.hadoop.hdfs.server.namenode.FSDirectory.updateCount(FSDirectory.java:1789)
            at org.apache.hadoop.hdfs.server.namenode.FSDirectory.updateCount(FSDirectory.java:1764)
            at org.apache.hadoop.hdfs.server.namenode.FSDirectory.addBlock(FSDirectory.java:357)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.saveAllocatedBlock(FSNamesystem.java:2847)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2508)
            at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:555)
            at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:387)

    》bin/hdfs dfsadmin -clrSpaceQuota /finance 将配额清除掉
    》bin/hdfs dfs -put readme1.txt /finance 这次上传就不会错了~

    [apache-nutch-1.2.rar 大约248M]
    bin/hdfs fsck /finance/apache-nutch-1.2.rar



    bin/hdfs fsck /finance/apache-nutch-1.2.rar -files -blocks -locations 将列出block在哪个位置


    
    
    bin/hdfs dfs -stat "%o %r" /finance/apache-nutch-1.2.rar                  查看某个文件有几个副本,及每个block的大小~~~
  • 相关阅读:
    git
    sublime text
    WIX安装图文并茂简易说明
    C#利用Lambda和Expression实现数据的动态绑定
    Ubuntu/CentOS下如何将普通用户提升到root权限
    虚拟机安装CentOS6.4用“桥接:直接连接到物理网线”不能上网的原因及解决方法
    C#利用lambda表达式将函数作为参数或属性跨类传递
    直接将XML存入到SQL中(SQL2008)
    C# 类中继承接口的属性
    提高开发效率的Visual Studio 2010使用技巧
  • 原文地址:https://www.cnblogs.com/i80386/p/3580107.html
Copyright © 2011-2022 走看看