MySQL Test Suite使用
MySQL自动测试套件（The MySQL Test Suite）用于对MySQL程序进行测试，包括各种功能与存储引擎。包含于MySQL与MariaDB版本代码中，位于mysql-test目录下。总体测试脚本为mysql-test-run.pl。该perl脚本通过调用各种已有的测试脚本，将测试结果与预置的result文件做对比来判断测试是否通过。（任何一点差异都会导致测试失败，包括预期之外的warning）
一、主要目录介绍
-
- include
包含.inc文件，用于测试开始时判断是否满足测试条件，在测试用通过source命令引入
如hava_innodb.inc:
|
1
2
3
4
|
if (`SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.ENGINES WHERE engine = 'innodb' AND support IN ('YES', 'DEFAULT', 'ENABLED')`)
{
--skip Test requires InnoDB.
}
|
如不满足，则会跳过需要该验证的测试。
-
- suite
包含所有测试suite，每一个suite为一个测试用例集，具体用例集在2节介绍
-
- t
1、测试文件
例：1st.test
|
1
2
3
4
5
|
#
# Check that we haven't any strange new tables or databases
#
show databases;
show tables in mysql;
|
上面内容即为1st这个测试用例的测试脚本
2、配置文件
例：innodb_bug53674-master.opt
–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed
为该测试需要的特殊配置项
3、脚本文件，在测试开始之前执行的脚本
例：rpl_misc_functions-slave.sh
rm -f $MYSQLTEST_VARDIR/master-data/test/rpl_misc_functions.outfile
-
- r
结果文件，判断是否通过的依据
使用上面的例子：1st.result
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
|
show databases;
Database
information_schema
mtr
mysql
performance_schema
test
show tables in mysql;
Tables_in_mysql
column_stats
columns_priv
db
event
func
general_log
gtid_slave_pos
help_category
help_keyword
help_relation
help_topic
host
index_stats
innodb_index_stats
innodb_table_stats
plugin
proc
procs_priv
proxies_priv
roles_mapping
servers
slow_log
table_stats
tables_priv
time_zone
time_zone_leap_second
time_zone_name
time_zone_transition
time_zone_transition_type
user
|
执行完1st用例会严格比对该文件。完全一致则测试通过。
-
- std_date
有些测试需要用到一些标准数据。
-
- var
测试默认使用该文件夹下的my.cnf文件
二、测试用例集
./mysql-test-run.pl不带任何参数表示执行所有测试用例集。使用—suite=suitename来指定单独执行suitename目录下的所有测试用例。
测试用例集包括：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
|
main：主要测试用例
archive
binlog
csv
encryption：加密功能
engines
federated
funcs_1：额外功能（包括视图、存储过程、INFORMATION_SCHEMA等）
funcs_2：额外功能（字符集等）
galera
galera_3nodes
handler
heap
innodb
innodb_fts：innodb全文索引
innodb_zip：innodb压缩
jp：日本字符相关测试
large_tests：
maria
mtr
mtr2
multi_source
ndb
optimizer_unfixed_bugs
parts
percona
perfschema
perfschema_stress
plugins
roles
rpl
storage_engine
stress
sys_vars
unit
vcol
wsrep
|
三、主要参数
force：默认情况下，只要遇到一个用例出错，测试程序就会退出，在指定force的情况下，测试程序会继续执行下面的测试（但是最多发现10个错误还是会退出）
suite：指定使用的测试suite
do-test：会以输入的字符串进行匹配用例执行
skip-test：会以输入的字符串进行匹配跳过用例执行
big-test：执行标记为big的测试用例。因为用例较大、耗时较长，标记为big的用例默认不会执行。输入两遍big-test则只执行标记为big的测试用例
recored：重新生成结果文件
mysqld：传递启动参数给mysqld，每个参数需要分别指定一个mysqld
其他还有很多选项，可以使用./mtr –help查看
如果需要在服务启动前执行一些脚本，可以写在 t/testname.sh文件中，由mtr自动执行。
四、测试举例
例1：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
|
[lzk@redhat64-26 mysql-test]$ ./mtr --force --big-test --suite=archive
Logging: ./mtr --force --big-test --suite=archive
vardir: /home/lzk/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/lzk/mysql-test/var'...
Checking supported features...
MariaDB Version 10.1.12-MariaDB
- SSL connections supported
Using suites: archive
Collecting tests...
Installing system database...
==============================================================================
TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
archive.archive_debug [ skipped ] Requires debug build
archive.partition_archive [ pass ] 74
archive.archive_no_symlink [ pass ] 5
archive.archive-big [ pass ] 298619
archive.archive [ pass ] 417
archive.archive_bitfield [ pass ] 37
archive.archive_gis [ pass ] 66
archive.archive_symlink [ pass ] 11
archive.discover [ pass ] 47
archive.discover_5438 [ pass ] 7
archive.mysqlhotcopy_archive [ pass ] 1035
archive.repair [ pass ] 3
archive.archive_plugin [ pass ] 6
--------------------------------------------------------------------------
The servers were restarted 3 times
Spent 300.327 of 309 seconds executing testcases
Completed: All 12 tests were successful.
1 tests were skipped, 0 by the test itself.
|
可以看出共完成12个测试用例，全部通过。1个用例必须在debug模式下测试，所以跳过。测试过程中服务重启3次。测试默认使用16000-16019端口。
例2：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
[lzk@redhat64-26 mysql-test]$ ./mtr binlog_innodb_row
Logging: ./mtr binlog_innodb_row
vardir: /home/lzk/mysql-test/var
Checking leftover processes...
- found old pid 28610 in 'mysqld.1.pid', killing it...
process did not exist!
Removing old var directory...
Creating var directory '/home/lzk/mysql-test/var'...
Checking supported features...
MariaDB Version 10.1.12-MariaDB
- SSL connections supported
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550)
Collecting tests...
Installing system database...
==============================================================================
TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
binlog.binlog_innodb_row 'innodb_plugin,row' [ pass ] 3361
binlog.binlog_innodb_row 'row,xtradb' [ pass ] 3349
--------------------------------------------------------------------------
The servers were restarted 1 times
Spent 6.710 of 17 seconds executing testcases
Completed: All 2 tests were successful.
|
每个测试结束前，mtr会检索error日志，如果发现warning或error，则测试失败。
再来看两个失败用例：
（截取部分日志）
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
innodb.innodb_bug53674 'innodb_plugin' [ fail ] Found warnings/errors in server log file!
Test ended at 2016-03-24 20:57:36
line
2016-03-24 20:57:35 7fd0fc6fc7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html.
^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err
ok
- saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/'
innodb.innodb_bug53674 'xtradb' [ fail ] Found warnings/errors in server log file!
Test ended at 2016-03-24 20:57:37
line
2016-03-24 20:57:36 7f274c58b7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html.
^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err
ok
- saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/'
|
分析日志可以看出是在测试过程中在错误日志中发现了warnings/errors。原因是使用了废弃的参数innodb_locks_unsafe_for_binlog导致warning。
找到并修改配置文件：innodb_bug53674-master.opt
将–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed
修改为：–transaction-isolation=READ-COMMITTED –binlog-format=mixed
重新测试：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
[lzk@redhat64-26 mysql-test]$ ./mtr innodb_bug53674
Logging: ./mtr innodb_bug53674
vardir: /home/lzk/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/lzk/mysql-test/var'...
Checking supported features...
MariaDB Version 10.1.12-MariaDB
- SSL connections supported
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550)
Collecting tests...
Installing system database...
==============================================================================
TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
innodb.innodb_bug53674 'innodb_plugin' [ pass ] 211
innodb.innodb_bug53674 'xtradb' [ pass ] 205
--------------------------------------------------------------------------
The servers were restarted 1 times
Spent 0.416 of 10 seconds executing testcases
Completed: All 2 tests were successful.
|
不再有warning，测试通过。