zoukankan      html  css  js  c++  java
  • 【数据库测试工具】认识Sysbench

    本文基于课堂PPT笔记整理,主要介绍一下Sysbench及其简单使用,实验代码部分在代码中有重点注释,不另作说明。

    一,基准测试

    什么是数据库的基准测试?

    数据库的基准测试是对数据库的性能指标进行定量的、可复现的、可对比的测试。

    数据库的基准测试有何作用?

    对数据库的基准测试的作用,就是分析在当前的配置下(包括硬件配置、OS、数据库设置等),数据库的性能表现,从而找出MySQL的性能阈值,并根据实际系统的要求调整配置。

    基准测试可以理解为针对系统的一种压力测试。但基准测试不关心业务逻辑,更加简单、直接、易于测试,数据可以由工具生成,不要求真实;而压力测试一般考虑业务逻辑(如购物车业务),要求真实的数据。

    数据库的基准测试有哪些测试指标?

    1)TPS/QPS:衡量吞吐量;

    2)响应时间:包括平均响应时间、最小响应时间、最大响应时间、时间百分比等,其中时间百分比参考意义较大,如前95%的请求的最大响应时间;

    3)并发量:同时处理的查询请求的数量。

    二,Sysbench简介

    Sysbench是跨平台的基准测试工具,支持多线程,支持多种数据库;主要包括以下几种测试:

    cpu性能

    磁盘io性能

    调度程序性能

    内存分配及传输速度

    POSIX线程性能

    数据库性能(OLTP基准测试)

    Sysbench基本命令一览表:

    ☁  ~  sysbench --help
    Usage:
      sysbench [options]... [testname] [command]
    
    Commands implemented by most tests: prepare run cleanup help
    
    General options:
      --threads=N                     number of threads to use [1]
      --events=N                      limit for total number of events [0]
      --time=N                        limit for total execution time in seconds [10]
      --forced-shutdown=STRING        number of seconds to wait after the --time limit before forcing shutdown, or 'off' to disable [off]
      --thread-stack-size=SIZE        size of stack per thread [64K]
      --rate=N                        average transactions rate. 0 for unlimited rate [0]
      --report-interval=N             periodically report intermediate statistics with a specified interval in seconds. 0 disables intermediate reports [0]
      --report-checkpoints=[LIST,...] dump full statistics and reset all counters at specified points in time. The argument is a list of comma-separated values representing the amount of time in seconds elapsed from start of test when report checkpoint(s) must be performed. Report checkpoints are off by default. []
      --debug[=on|off]                print more debugging info [off]
      --validate[=on|off]             perform validation checks where possible [off]
      --help[=on|off]                 print help and exit [off]
      --version[=on|off]              print version and exit [off]
      --config-file=FILENAME          File containing command line options
      --tx-rate=N                     deprecated alias for --rate [0]
      --max-requests=N                deprecated alias for --events [0]
      --max-time=N                    deprecated alias for --time [0]
      --num-threads=N                 deprecated alias for --threads [1]
    
    Pseudo-Random Numbers Generator options:
      --rand-type=STRING random numbers distribution {uniform,gaussian,special,pareto} [special]
      --rand-spec-iter=N number of iterations used for numbers generation [12]
      --rand-spec-pct=N  percentage of values to be treated as 'special' (for special distribution) [1]
      --rand-spec-res=N  percentage of 'special' values to use (for special distribution) [75]
      --rand-seed=N      seed for random number generator. When 0, the current time is used as a RNG seed. [0]
      --rand-pareto-h=N  parameter h for pareto distribution [0.2]
    
    Log options:
      --verbosity=N verbosity level {5 - debug, 0 - only critical messages} [3]
    
      --percentile=N       percentile to calculate in latency statistics (1-100). Use the special value of 0 to disable percentile calculations [95]
      --histogram[=on|off] print latency histogram in report [off]
    
    General database options:
    
      --db-driver=STRING  specifies database driver to use ('help' to get list of available drivers) [mysql]
      --db-ps-mode=STRING prepared statements usage mode {auto, disable} [auto]
      --db-debug[=on|off] print database-specific debug information [off]
    
    
    Compiled-in database drivers:
      mysql - MySQL driver
    
    mysql options:
      --mysql-host=[LIST,...]          MySQL server host [localhost]
      --mysql-port=[LIST,...]          MySQL server port [3306]
      --mysql-socket=[LIST,...]        MySQL socket
      --mysql-user=STRING              MySQL user [sbtest]
      --mysql-password=STRING          MySQL password []
      --mysql-db=STRING                MySQL database name [sbtest]
      --mysql-ssl[=on|off]             use SSL connections, if available in the client library [off]
      --mysql-ssl-cipher=STRING        use specific cipher for SSL connections []
      --mysql-compression[=on|off]     use compression, if available in the client library [off]
      --mysql-debug[=on|off]           trace all client library calls [off]
      --mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205]
      --mysql-dry-run[=on|off]         Dry run, pretend that all MySQL client API calls are successful without executing them [off]
    
    Compiled-in tests:
      fileio - File I/O test
      cpu - CPU performance test
      memory - Memory functions speed test
      threads - Threads subsystem performance test
      mutex - Mutex performance test
    
    See 'sysbench <testname> help' for a list of options for each test.
    

    三,Mac安装使用Sysbench

    ☁  ~  brew install sysbench
    # 添加路径
    ☁  ~  export LDFLAGS="-L/usr/local/opt/mysql@5.7/lib"
    ☁  ~  export CPPFLAGS="-I/usr/local/opt/mysql@5.7/include"
    # 查看版本
    ☁  ~  sysbench --version
    # 测试自带lua脚本
    ☁  ~  sysbench --test=/usr/local/Cellar/sysbench/1.0.19/share/sysbench/tests/include/oltp_legacy/oltp.lua
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    ☁  ~  sysbench /usr/local/Cellar/sysbench/1.0.19/share/sysbench/tests/include/oltp_legacy/oltp.lua
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    # 测试cpu
    ☁  ~  sysbench cpu --cpu-max-prime=20000 --threads=3 run
    # 测试file,run是执行正式的测试,prepare是为测试提前准备数据,cleanup是在测试完成后对数据库进行清理
    ☁  ~  sysbench fileio --threads=16 --file-total-size=1G --file-test-mode=rndrw prepare
    ☁  ~  sysbench fileio --threads=16 --file-total-size=1G --file-test-mode=rndrw run
    ☁  ~  sysbench fileio --threads=16 --file-total-size=1G --file-test-mode=rndrw cleanup
    # 测试threads
    ☁  ~  sysbench threads --threads=64 --thread-yields=100 --thread-locks=2 run
    # 测试memory
    ☁  ~  sysbench memory --memory-block-size=8k --memory-total-size=40G run
    # OLTP测试
    ☁  ~  sysbench /usr/local/Cellar/sysbench/1.0.19/share/sysbench/tests/include/oltp_legacy/oltp.lua --mysql-table-engine=innodb --oltp-tables-count=3 --oltp-table-size=100000 --mysql-user=root --mysql-password=xxxxxxx --mysql-socket=/tmp/mysql.sock prepare
    ☁  ~  sysbench /usr/local/Cellar/sysbench/1.0.19/share/sysbench/tests/include/oltp_legacy/oltp.lua --mysql-table-engine=innodb --oltp-tables-count=3 --oltp-table-size=100000 --mysql-user=root --mysql-password=xxxxxx --mysql-socket=/tmp/mysql.sock --max-time=60 --max-requests=0 --num-threads=8 --report-interval=10 run
    WARNING: --num-threads is deprecated, use --threads instead
    WARNING: --max-time is deprecated, use --time instead
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    
    Running the test with following options:
    Number of threads: 8
    Report intermediate results every 10 second(s)
    Initializing random number generator from current time
    
    
    Initializing worker threads...
    
    Threads started!
    
    [ 10s ] thds: 8 tps: 658.76 qps: 13182.94 (r/w/o: 9229.30/2635.43/1318.21) lat (ms,95%): 26.68 err/s: 0.00 reconn/s: 0.00
    [ 20s ] thds: 8 tps: 717.41 qps: 14350.05 (r/w/o: 10045.00/2870.13/1434.91) lat (ms,95%): 22.69 err/s: 0.00 reconn/s: 0.00
    [ 30s ] thds: 8 tps: 745.71 qps: 14914.47 (r/w/o: 10439.89/2983.15/1491.43) lat (ms,95%): 21.11 err/s: 0.00 reconn/s: 0.00
    [ 40s ] thds: 8 tps: 754.39 qps: 15085.46 (r/w/o: 10560.23/3016.45/1508.78) lat (ms,95%): 21.11 err/s: 0.00 reconn/s: 0.00
    [ 50s ] thds: 8 tps: 687.47 qps: 13749.32 (r/w/o: 9624.39/2749.98/1374.94) lat (ms,95%): 24.83 err/s: 0.00 reconn/s: 0.00
    [ 60s ] thds: 8 tps: 620.78 qps: 12417.29 (r/w/o: 8692.61/2483.12/1241.56) lat (ms,95%): 27.17 err/s: 0.00 reconn/s: 0.00
    SQL statistics:
        queries performed:
            read:                            585984
            write:                           167424
            other:                           83712
            total:                           837120
        transactions:                        41856  (697.33 per sec.)
        queries:                             837120 (13946.58 per sec.)
        ignored errors:                      0      (0.00 per sec.)
        reconnects:                          0      (0.00 per sec.)
    
    General statistics:
        total time:                          60.0215s
        total number of events:              41856
    
    Latency (ms):
             min:                                    3.54
             avg:                                   11.47
             max:                                  138.32
             95th percentile:                       23.95
             sum:                               479897.13
    
    Threads fairness:
        events (avg/stddev):           5232.0000/18.17
        execution time (avg/stddev):   59.9871/0.01
    ☁  ~  sysbench /usr/local/Cellar/sysbench/1.0.19/share/sysbench/tests/include/oltp_legacy/oltp.lua --mysql-table-engine=innodb --oltp-tables-count=3 --oltp-table-size=100000 --mysql-user=root --mysql-password=xxxxxxx --mysql-socket=/tmp/mysql.sock --report-interval=10 cleanup
    

    四,安装并使用TPCC-MYSQL

    Tpcc-mysql默认自带的数据库比较大,约10G,请实验前保证磁盘空间充足。

    # 下载源码
    ☁  tools git clone https://github.com/Percona-Lab/tpcc-mysql.git
    # 配置mysql_config
    ☁  ~  sudo ln -s /usr/local/opt/mysql@5.7/bin/mysql_config /usr/local/bin/mysql_config
    # Bulid binaries
    ☁  tools  cd tpcc-mysql/src/
    ☁  src [master] ⚡  make
    # Load data
    ☁  ~  mysqladmin -uroot -p create tpcc1000
    ☁  ~  mysql -uroot -p  tpcc1000 < /home/tools/tpcc-mysql/create_table.sql
    ☁  ~  mysql -uroot -p  tpcc1000 < /home/tools/tpcc-mysql/add_fkey_idx.sql
    ☁  tpcc-mysql [master] ⚡  ./tpcc_load -h127.0.0.1 -d tpcc1000 -uroot -pxxxxxx -w 100
    # Start benchmark
    ☁  tpcc-mysql [master] ⚡  ./tpcc_start -h 127.0.0.1 -p 3306 -d tpcc1000 -uroot -p "xxxxxxx" -w 100 -c 10 -r 100 -l 300 -i 20 -f /var/log/tpcc_mysql.log -t /var/log/tpcc_mysql.rtx
    ....
    <Constraint Check> (all must be [OK])
     [transaction percentage]
            Payment: 43.48% (>=43.0%) [OK]
       Order-Status: 4.35% (>= 4.0%) [OK]
           Delivery: 4.35% (>= 4.0%) [OK]
        Stock-Level: 4.35% (>= 4.0%) [OK]
     [response time (at least 90% passed)]
          New-Order: 0.00%  [NG] *
            Payment: 30.49%  [NG] *
       Order-Status: 34.91%  [NG] *
           Delivery: 12.76%  [NG] *
        Stock-Level: 0.00%  [NG] *
    
    <TpmC>
                     4061.400 TpmC
    

    五,遇到的问题及其解决方案

    问题描述:ld: library not found for -lrt

    解决方案:在src/Makefile中去掉 lrt,产生的错误按照提示去解决即可。

    LIBS=       `mysql_config --libs_r` # -lrt
    
  • 相关阅读:
    Codevs 2296 仪仗队 2008年省队选拔赛山东
    Codevs 1535 封锁阳光大学
    Codevs 1069 关押罪犯 2010年NOIP全国联赛提高组
    Codevs 1218 疫情控制 2012年NOIP全国联赛提高组
    Codevs 1684 垃圾陷阱
    洛谷 P1108 低价购买
    Vijos P1325桐桐的糖果计划
    Codevs 3289 花匠 2013年NOIP全国联赛提高组
    Codevs 2611 观光旅游(floyed最小环)
    C语言基础之彩色版C语言(内含linux)
  • 原文地址:https://www.cnblogs.com/zhangbc/p/12684483.html
Copyright © 2011-2022 走看看