zoukankan      html  css  js  c++  java
  • hadoop 2012.07.24 读写测试

    namenode 独立虚拟机:1G内存

    datanode1、datanode2 虚拟于同一塔式服务器,内存各1G

     

    ----- TestDFSIO ----- : write
    Date & time: Tue Jul 24 17:15:06 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 16.458439970308973
    Average IO rate mb/sec: 17.915302276611328
    IO rate std deviation: 5.502547439324805
    Test exec time sec: 702.77

    关闭数据库后测试

    Date & time: Tue Jul 24 18:16:42 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 22.132315836999922
    Average IO rate mb/sec: 24.76445960998535
    IO rate std deviation: 8.614986952186385
    Test exec time sec: 543.453


    两台虚拟机作为datanode部署在一个节点上,在做写入测试时报Too many fetch-failures错误,这是因为读写过于频繁导致datanode与namenode通信异常导致的,移除其中一个datanode后再进行读写测试,就不再报联系不上的错误了。

    移除一台datanode后测试结果:

    ----- TestDFSIO ----- : write
    Date & time: Tue Jul 24 18:31:06 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 14.386874566595404
    Average IO rate mb/sec: 14.926310539245605
    IO rate std deviation: 2.5935396939344066
    Test exec time sec: 456.993

    两台建立与独立机器上的虚拟机,其中有一个datanode没有配置hosts文件,导致DiskErrorException: Could not find taskTracker

    ----- TestDFSIO ----- : write
    Date & time: Thu Jul 26 01:00:34 PDT 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 54.977376809442916
    Average IO rate mb/sec: 64.9064712524414

    IO rate std deviation: 27.569772802264755
    Test exec time sec: 346.016

    两台建立与独立机器上的虚拟机,正常配置datanode的hosts文件后

    ----- TestDFSIO ----- : write
    Date & time: Thu Jul 26 16:28:38 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 69.33322240017749
    Average IO rate mb/sec: 80.34981536865234
    IO rate std deviation: 31.796870275012598
    Test exec time sec: 333.343

  • 相关阅读:
    OSG嵌入QT(QT界面使用Qt Designer编辑)
    C++ Makefile文件编写
    cbp2make使用
    for循环之后的return
    C++引用形参,函数返回多个值
    个人感悟之代理模式
    个人感悟之单例模式
    个人感悟之简单工厂模式-工厂方法模式-抽象工厂模式
    Java设计模式
    (转)java之Spring(IOC)注解装配Bean详解
  • 原文地址:https://www.cnblogs.com/MorZe/p/2606889.html
Copyright © 2011-2022 走看看