zoukankan      html  css  js  c++  java
  • hadoop 2012.07.24 读写测试

    namenode 独立虚拟机:1G内存

    datanode1、datanode2 虚拟于同一塔式服务器,内存各1G

     

    ----- TestDFSIO ----- : write
    Date & time: Tue Jul 24 17:15:06 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 16.458439970308973
    Average IO rate mb/sec: 17.915302276611328
    IO rate std deviation: 5.502547439324805
    Test exec time sec: 702.77

    关闭数据库后测试

    Date & time: Tue Jul 24 18:16:42 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 22.132315836999922
    Average IO rate mb/sec: 24.76445960998535
    IO rate std deviation: 8.614986952186385
    Test exec time sec: 543.453


    两台虚拟机作为datanode部署在一个节点上,在做写入测试时报Too many fetch-failures错误,这是因为读写过于频繁导致datanode与namenode通信异常导致的,移除其中一个datanode后再进行读写测试,就不再报联系不上的错误了。

    移除一台datanode后测试结果:

    ----- TestDFSIO ----- : write
    Date & time: Tue Jul 24 18:31:06 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 14.386874566595404
    Average IO rate mb/sec: 14.926310539245605
    IO rate std deviation: 2.5935396939344066
    Test exec time sec: 456.993

    两台建立与独立机器上的虚拟机,其中有一个datanode没有配置hosts文件,导致DiskErrorException: Could not find taskTracker

    ----- TestDFSIO ----- : write
    Date & time: Thu Jul 26 01:00:34 PDT 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 54.977376809442916
    Average IO rate mb/sec: 64.9064712524414

    IO rate std deviation: 27.569772802264755
    Test exec time sec: 346.016

    两台建立与独立机器上的虚拟机,正常配置datanode的hosts文件后

    ----- TestDFSIO ----- : write
    Date & time: Thu Jul 26 16:28:38 CST 2012
    Number of files: 10
    Total MBytes processed: 10000
    Throughput mb/sec: 69.33322240017749
    Average IO rate mb/sec: 80.34981536865234
    IO rate std deviation: 31.796870275012598
    Test exec time sec: 333.343

  • 相关阅读:
    WEB开发中合理选择图片格式
    Ext.ux.form.LovCombo bug修正
    Ext.grid.PropertyGrid 扩展
    BLOG代码高亮
    Box2D教程1创建碰撞世界
    Box2D教程2鼠标交互
    Box2D教程5碰撞检测
    Box2D教程3刚体绑定外观
    管窥HTML5
    Box2D教程4复杂刚体的复杂外观
  • 原文地址:https://www.cnblogs.com/MorZe/p/2606889.html
Copyright © 2011-2022 走看看