=========================================
The number of tests we have for MySQL Server are constantly growing. There is a need to group them in different ways so we can select what and where to run. We do this by using suites, either the default suite that we call "main" in mysql-test/t or one of the subdirs of mysql-test/suite.
As each test becomes more advanced it's also necessary to use different configurations for a particular test or suite. For example all the replication tests in suite/rpl need to be run with the server started in three different ways(three different configurations) to get full coverage. To avoid that the individual developer or the "one" running tests have to remember different parameters to run the tests with, we have made it possible to make each suite special. That means it can have different number of MySQL Servers and different settings of each server.
The current mysql-test-run.pl in MySQL 5.1 has:
- support for running tests from suite/ directory
- each suite can have some special setting with the use of a suite.opt file.
- each suite can make each test to be run more different configurations by adding a combinations file(see for example suite/rpl/). That file will cause all .test to be "multiplied" by the number of combinations and thus run several times with different server settings.
- All the standard suites to look for are added to mysql-test-run.pl to makerit as easy as possible for a developer to run the default set of tests.
We also have a new mysql-test-run.pl that is going into 5.1 and up very soon. it has been extended to also support:
- allowing each suite to have it's own my.cnf file(s), this way we can run tests with various settings of the server.
- that each .test has it's own
- to find suites in storage/
- to find a suite in any directory by simply giving the full path to where that suite is.
In the future we'll continue to make it possible for each suite to be even more special(in a pluggable way) as we find the need for it.
2 comments:
Timely reading for me as I've just been trying to get mysql-test-run.pl to work with a set of unit tests that I am devising to test some triggers and stored functions/procedures in a recent MySQL custom database.
While casting around for a unit testing framework, when I found mysqltest and mysql-test-run.pl, I thought "What could be better than to use what the developers of the engine are using themselves?"
My problem has been the paucity of examples of how to use mysql-test-run.pl for anything other than the tests of the engine that reside in the same directory structure. It seems that the perl script MUST run in the directory in which it is installed, and that you can not point it at your own directory of tests.
I just started looking at the Perl source to see how to create a new --base option to pass down to the call to mysqltest. I think this is the way to go. Do you agree? Can you advise?
1. Yes, it should be supported to point mysql-test-run.pl to your own directory(suite) with .test and .result files. This is included in the next version.
2. I've more or less resolved the issue with that you have to cd to mysql-test/ before you run mtr.pl, but it still has a check that you run it from mysql-test/ since we don't have time to really test any other ways to run it.
Don't think a --base is needed, but maybe you can describe what it should do?
Hope you find mysql-test-run.pl useful.
Post a Comment