zoukankan      html  css  js  c++  java
  • Ceph性能测试

    一.测试分类

    ceph的性能测试包含rados性能测试和rbd性能测试

    二.测试工具

    rados性能测试工具:使用ceph自带的rados bench工具、rados losd-gen工具
    rbd性能测试工具:rbd bench-write进行块设备写性能测试,fio+rbd ioengine测试、fio+libaio测试

    三.测试示例

    3.1 rados性能测试之rados bench

    3.1.1 工具的命令行语法

    rados bench -p <pool_name> <seconds> <write|seq|rand> -b <block size> -t --no-cleanup
    
    • pool_name: 测试的存储池名称
    • seconds: 测试时间,以秒为单位
    • <write|seq|rand>:操作的模式,write:写 seq:顺序读 rand:随机读
    • -b:block size即块大小,默认为4M
    • -t:读/写并行数,默认为16
    • --no-cleanup:表示测试完成后不删除测试用数据,在做读测试之前,需要使用该参数运行一遍写测试来产生数据,在全部测试完毕后可以运行rados -p <pool_name> cleanup 来清理数据

    3.1.2 执行写测试

    • 创建一个测试使用的存储池test
    [root@openstack01 ~]# docker exec ceph_mon ceph osd pool create test 512 512
    
    • 执行写测试,测试时间设置为60s
    (ceph-mon)[root@openstack01 ~]# rados bench -p test 60 write --no-cleanup
    hints = 1
    Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 60 seconds or 0 objects
    Object prefix: benchmark_data_openstack01_603
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
        0       0         0         0         0         0           -           0
        1      16        16         0         0         0           -           0
        2      16        16         0         0         0           -           0
        3      16        24         8   10.6653   10.6667     2.66058     2.22005
        4      16        27        11   10.9986        12     3.19831     2.45947
        5      16        29        13   10.3986         8     2.54069     2.61219
        6      16        38        22   14.6647        36     2.52673     3.23277
        7      16        43        27   15.4265        20     4.06122     3.28708
        8      16        46        30    14.998        12     3.23905     3.27569
        9      16        47        31   13.7759         4     2.94027     3.26487
       10      16        50        34   13.5982        12     3.41596     3.31326
       11      16        55        39   14.1799        20      5.3298     3.45411
       12      16        58        42   13.9981        12     5.90129     3.60335
       13      16        59        43    13.229         4     6.06364     3.66057
       14      16        61        45   12.8554         8     5.25054     3.74845
       15      16        64        48   12.7982        12     8.46896     3.84977
       16      16        65        49   12.2483         4     7.87838     3.93199
       17      16        68        52   12.2336        12     7.36673     4.06689
       18      16        72        56   12.4427        16        7.06      4.2234
       19      16        75        59   12.4193        12     7.78165     4.30382
    2018-10-11 11:57:52.537579 min lat: 1.89714 max lat: 8.54175 avg lat: 4.35334
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
       20      16        79        63   12.5982        16     6.90524     4.35334
       21      16        83        67   12.7601        16     5.97492     4.47472
       22      16        85        69   12.5437         8     5.49733     4.50754
       23      16        86        70   12.1722         4      4.1921     4.50304
       24      16        87        71   11.8317         4     6.14773      4.5262
       25      16        90        74   11.8383        12     5.11989      4.5118
       26      16        91        75   11.5368         4      5.1742     4.52063
       27      16        95        79   11.7021        16     8.31085     4.62613
       28      16        96        80    11.427         4      7.6217     4.66358
       29      16        96        80   11.0329         0           -     4.66358
       30      16        96        80   10.6652         0           -     4.66358
       31      16        96        80   10.3211         0           -     4.66358
       32      16        96        80   9.99859         0           -     4.66358
       33      16        96        80   9.69558         0           -     4.66358
       34      16        96        80   9.41042         0           -     4.66358
       35      16       101        85   9.71291   2.85714      8.7633     4.95709
       36      16       101        85   9.44309         0           -     4.95709
       37      16       103        87   9.40406         4     17.9027      5.1727
       38      16       105        89   9.36708         8     18.0327     5.37878
       39      16       105        89   9.12691         0           -     5.37878
    2018-10-11 11:58:12.540426 min lat: 1.89714 max lat: 18.0327 avg lat: 5.37878
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
       40      16       105        89   8.89874         0           -     5.37878
       41      16       105        89   8.68167         0           -     5.37878
       42      16       106        90   8.57019         1     6.43809     5.39055
       43      16       114        98   9.11497        32     5.11341     6.40932
       44      16       115        99   8.99871         4     7.16406     6.41694
       45      16       115        99   8.79874         0           -     6.41694
       46      16       116       100   8.69442         2     11.4983     6.46775
       47      16       118       102   8.67962         8     9.64869     6.54381
       48      16       119       103   8.58212         4     5.13036     6.53009
       49      16       119       103   8.40697         0           -     6.53009
       50      16       120       104   8.31882         2     7.10649     6.53563
       51      16       125       109   8.54781        20     8.44788     6.79488
       52      16       130       114     8.768        20     9.57393     6.83936
       53      16       134       118    8.9044        16     10.3648     6.79498
       54      16       136       120   8.88764         8     9.32118     6.81407
       55      16       140       124   9.01692        16     2.17708     6.69443
       56      16       140       124    8.8559         0           -     6.69443
       57      16       143       127   8.91103         6     4.81556     6.68976
       58      16       146       130   8.96426        12     10.3827     6.68146
       59      16       146       130   8.81232         0           -     6.68146
    2018-10-11 11:58:32.543137 min lat: 1.63302 max lat: 23.564 avg lat: 6.68146
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
       60      16       146       130   8.66545         0           -     6.68146
       61      16       146       130   8.52339         0           -     6.68146
       62      16       146       130   8.38592         0           -     6.68146
       63      16       146       130   8.25281         0           -     6.68146
       64      16       146       130   8.12387         0           -     6.68146
       65      16       146       130   7.99889         0           -     6.68146
       66      16       146       130   7.87769         0           -     6.68146
       67      11       147       136   8.11828   2.66667     15.7406     6.94076
       68      11       147       136   7.99889         0           -     6.94076
       69      11       147       136   7.88297         0           -     6.94076
    Total time run:         69.107454
    Total writes made:      147
    Write size:             4194304
    Object size:            4194304
    Bandwidth (MB/sec):     8.50849
    Stddev Band       7.95831
    Max bandwidth (MB/sec): 36
    Min bandwidth (MB/sec): 0
    Average IOPS:           2
    Stddev IOPS:            2
    Max IOPS:               9
    Min IOPS:               0
    Average Latency(s):     7.44131
    Stddev Latency(s):      4.9394
    Max latency(s):         23.564
    Min latency(s):         1.63302
    

    上面数据Bandwidth为8.50849MB/sec,Average Latency为:7.44131s

    3.1.3 执行随机读测试

    (ceph-mon)[root@openstack01 ~]# rados bench -p test 60 rand
    hints = 1
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
        0       0         0         0         0         0           -           0
        1      16       152       136   543.849       544   0.0743183   0.0995761
        2      16       306       290   579.818       616  0.00970808    0.102821
        3      16       444       428   570.514       552   0.0405567    0.104487
        4      15       568       553   552.862       500  0.00818665    0.109911
        5      16       700       684   546.952       524    0.043003    0.111776
        6      16       836       820   546.446       544    0.104833    0.113024
        7      16       959       943   538.658       492   0.0117524    0.114411
        8      16      1079      1063   531.318       480   0.0381715    0.116316
        9      16      1199      1183   525.608       480   0.0552762    0.118574
       10      16      1293      1277   510.644       376   0.0362924    0.117607
       11      16      1423      1407   511.476       520   0.0142599    0.122025
       12      16      1566      1550   516.511       572    0.008037     0.12061
       13      16      1686      1670   513.694       480  0.00940001    0.119639
       14      16      1793      1777   507.568       428   0.0111657    0.123314
       15      16      1923      1907    508.39       520   0.0120602    0.123097
       16      16      2047      2031   507.611       496   0.0106408     0.12326
       17      16      2164      2148   505.277       468  0.00817543    0.123239
       18      16      2264      2248   499.426       400   0.0130886    0.125529
       19      15      2391      2376   500.083       512  0.00987636    0.125445
    2018-10-11 14:27:20.997424 min lat: 0.00644293 max lat: 1.0492 avg lat: 0.125502
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
       20      16      2516      2500   499.875       496  0.00979231    0.125502
       21      16      2642      2626   500.067       504    0.271557    0.125655
       22      16      2769      2753   500.424       508  0.00853595     0.12471
       23      16      2891      2875    499.88       488    0.358424    0.125615
       24      16      3009      2993   498.716       472    0.356571    0.125675
       25      16      3127      3111   497.645       472    0.452718    0.125959
       26      15      3269      3254     500.5       572    0.286575    0.125597
       27      16      3393      3377   500.182       492   0.0994856    0.125185
       28      16      3512      3496   499.316       476   0.0637264    0.125534
       29      16      3648      3632   500.853       544    0.307832    0.125653
       30      15      3791      3776   503.354       576  0.00929097    0.124856
       31      16      3919      3903   503.501       508  0.00955382    0.124689
       32      16      4045      4029   503.514       504    0.180285    0.124946
       33      15      4150      4135   501.103       424   0.0448866    0.125427
       34      16      4287      4271   502.361       544    0.363722    0.125243
       35      16      4394      4378   500.233       428  0.00944837    0.125241
       36      16      4512      4496   499.447       472   0.0106443       0.126
       37      16      4629      4613   498.591       468   0.0995126    0.126007
       38      16      4738      4722   496.941       436    0.267176    0.126746
       39      16      4869      4853   497.626       524    0.303769    0.126441
    2018-10-11 14:27:41.001727 min lat: 0.00644293 max lat: 1.0492 avg lat: 0.126309
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
       40      16      4999      4983   498.184       520   0.0542142    0.126309
       41      16      5119      5103   497.739       480  0.00888314    0.126283
       42      15      5245      5230   497.976       508   0.0360334    0.126255
       43      16      5380      5364   498.857       536    0.015746    0.126103
       44      16      5508      5492   499.152       512     0.22606    0.126146
       45      16      5626      5610   498.547       472   0.0110425    0.126163
       46      16      5759      5743   499.273       532    0.378206    0.126063
       47      16      5885      5869   499.372       504   0.0112606    0.125814
       48      15      6000      5985   498.634       464     0.42884    0.126265
       49      16      6127      6111    498.74       504    0.611502    0.126078
       50      16      6255      6239   499.004       512    0.100672      0.1262
       51      16      6370      6354   498.238       460    0.428983     0.12635
       52      16      6495      6479   498.271       500   0.0130692    0.126363
       53      16      6620      6604   498.302       500    0.160325    0.126399
       54      16      6748      6732   498.554       512   0.0384334    0.126252
       55      16      6885      6869   499.448       548  0.00900053     0.12599
       56      15      7016      7001   499.956       528   0.0435343    0.125842
       57      15      7136      7121   499.603       480   0.0516195    0.125927
       58      16      7243      7227   498.299       424    0.224335    0.126219
       59      16      7368      7352   498.326       500   0.0679909    0.126414
    2018-10-11 14:28:01.006114 min lat: 0.00644293 max lat: 1.0492 avg lat: 0.126286
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
       60      15      7502      7487   499.019       540   0.0106773    0.126286
    Total time run:       60.227155
    Total reads made:     7502
    Read size:            4194304
    Object size:          4194304
    Bandwidth (MB/sec):   498.247
    Average IOPS:         124
    Stddev IOPS:          10
    Max IOPS:             154
    Min IOPS:             94
    Average Latency(s):   0.126668
    Max latency(s):       1.0492
    Min latency(s):       0.00644293
    

    上面数据Bandwidth为498.247MB/sec,Average Latency为:0.126668s

    3.1.4 执行顺序读测试

    (ceph-mon)[root@openstack01 ~]# rados bench -p test 60 seq
    hints = 1
      sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
        0       0         0         0         0         0           -           0
        1      16       147       131   523.861       524   0.0094258   0.0971213
    Total time run:       1.199266
    Total reads made:     147
    Read size:            4194304
    Object size:          4194304
    Bandwidth (MB/sec):   490.3
    Average IOPS:         122
    Stddev IOPS:          0
    Max IOPS:             131
    Min IOPS:             131
    Average Latency(s):   0.126695
    Max latency(s):       0.612257
    Min latency(s):       0.00736062
    

    上面数据Bandwidth为490.3MB/sec,Average Latency为:0.126695s

    3.2 rbd性能测试

  • 相关阅读:
    【MySQL】【2】数字排序问题
    【MySQL】【1】表中存在重复记录,删除保留其中一条
    poj 1811 Prim test
    Yours 的博客开张啦!
    P1044
    P1103
    hustwinter1-B
    hustwinterC
    hustwinter1-A
    hdu 2138 How many prime numbers
  • 原文地址:https://www.cnblogs.com/yuhaohao/p/9772242.html
Copyright © 2011-2022 走看看