Table of Contents
This chapter describes a lot of things that you need to know when working on the MySQL code. If you plan to contribute to MySQL development, want to have access to the bleeding-edge in-between versions code, or just want to keep track of development, follow the instructions in the section called “Installing from the Development Source Tree”. If you are interested in MySQL internals, you should also subscribe to our internals mailing list. This list is relatively low traffic. For details on how to subscribe, please see the section called “The MySQL Mailing Lists”. All developers at MySQL AB are on the internals list and we help other people who are working on the MySQL code. Feel free to use this list both to ask questions about the code and to send patches that you would like to contribute to the MySQL project!
The MySQL server creates the following threads:
The TCP/IP connection thread handles all connection requests and creates a new dedicated thread to handle the authentication and SQL query processing for each connection.
On Windows NT there is a named pipe handler thread that does the same work as the TCP/IP connection thread on named pipe connect requests.
The signal thread handles all signals. This thread also normally handles alarms and calls process_alarm() to force timeouts on connections that have been idle too long.
If mysqld is compiled with -DUSE_ALARM_THREAD, a dedicated thread that handles alarms is created. This is only used on some systems where there are problems with sigwait() or if you want to use the thr_alarm() code in your application without a dedicated signal handling thread.
If one uses the --flush_time=# option, a dedicated thread is created to flush all tables at the given interval.
Every connection has its own thread.
Every different table on which one uses INSERT DELAYED gets its own thread.
If you use --master-host, a slave replication thread will be started to read and apply updates from the master.
mysqladmin processlist only shows the connection, INSERT DELAYED, and replication threads.
Until recently, our main full-coverage test suite was based on proprietary customer data and for that reason has not been publicly available. The only publicly available part of our testing process consisted of the crash-me test, a Perl DBI/DBD benchmark found in the sql-bench directory, and miscellaneous tests located in tests directory. The lack of a standardized publicly available test suite has made it difficult for our users, as well developers, to do regression tests on the MySQL code. To address this problem, we have created a new test system that is included in Unix source distributions and binary distributions starting with Version 3.23.29. The tests can be run under Unix, or on Windows in the Cygwin environment if the server has been compiled under Cygwin. They cannot be run in a native Windows environment currently.
The current set of test cases doesn't test everything in MySQL, but it should catch most obvious bugs in the SQL processing code, OS/library issues, and is quite thorough in testing replication. Our eventual goal is to have the tests cover 100% of the code. We welcome contributions to our test suite. You may especially want to contribute tests that examine the functionality critical to your system, because this will ensure that all future MySQL releases will work well with your applications.
The test system consist of a test language interpreter (mysqltest), a shell script to run all tests(mysql-test-run), the actual test cases written in a special test language, and their expected results. To run the test suite on your system after a build, type make test or mysql-test/mysql-test-run from the source root. If you have installed a binary distribution, cd to the install root (eg. /usr/local/mysql), and do scripts/mysql-test-run. All tests should succeed. If not, you should try to find out why and report the problem if this is a bug in MySQL. See the section called “Reporting Bugs in the MySQL Test Suite”.
From MySQL 4.1 on, if you have a copy of mysqld running on the machine where you want to run the test suite you do not have to stop it, as long as it is not using ports 9306 and 9307. If one of those ports is taken, you should edit mysql-test-run and change the values of the master and/or slave port to one that is available.
Before MySQL 4.1, mysql-test-run does not try to run its own server by default but tries to use your currently running server. To override this and cause mysql-test-run to start its own server, run it with with the --local option.
You can run one individual test case with mysql-test/mysql-test-run test_name.
If one test fails, you should test running mysql-test-run with the --force option to check whether any other tests fail.
You can use the mysqltest language to write your own test cases. Unfortunately, we have not yet written full documentation for it. You can, however, look at our current test cases and use them as an example. The following points should help you get started:
The tests are located in mysql-test/t/*.test
A test case consists of ; terminated statements and is similar to the input of mysql command-line client. A statement by default is a query to be sent to MySQL server, unless it is recognized as internal command (eg. sleep).
All queries that produce results—for example, SELECT, SHOW, EXPLAIN, etc., must be preceded with @/path/to/result/file. The file must contain the expected results. An easy way to generate the result file is to run mysqltest -r < t/test-case-name.test from the mysql-test directory, and then edit the generated result files, if needed, to adjust them to the expected output. In that case, be very careful about not adding or deleting any invisible characters — make sure to only change the text and/or delete lines. If you have to insert a line, make sure that the fields are separated by a hard tab, and that there is a hard tab at the end. You may want to use od -c to make sure that your text editor has not messed anything up during edit. We hope that you will never have to edit the output of mysqltest -r as you only have to do it when you find a bug.
To be consistent with our setup, you should put your result files in the mysql-test/r directory and name them test_name.result. If the test produces more than one result, you should use test_name.a.result, test_name.b.result, etc.
If a statement returns an error, you should specify it with --error error-number on the line before the statement. The error number can be a list of possible error numbers separated by ‘,’.
If you are writing a replication test case, you should on the first line of the test file, put source include/master-slave.inc;. To switch between master and slave, use connection master; and connection slave;. If you need to do something on an alternate connection, you can do connection master1; for the master, and connection slave1; for the slave.
If you need to do something in a loop, you can use something like this:
let $1=1000;
while ($1)
{
# do your queries here
dec $1;
}
To sleep between queries, use the sleep command. It supports fractions of a second, so you can use sleep 1.3;, for example, to sleep 1.3 seconds.
To run the slave with additional options for your test case, put them in the command-line format in mysql-test/t/test_name-slave.opt. For the master, put them in mysql-test/t/test_name-master.opt.
If you have a question about the test suite, or have a test case to contribute, send an email message to the MySQL internals mailing list. See the section called “The MySQL Mailing Lists”. As this list does not accept attachments, you should ftp all the relevant files to: ftp://ftp.mysql.com/pub/mysql/upload/
If your MySQL version doesn't pass the test suite you should do the following:
Don't send a bug report before you have found out as much as possible of what when wrong! When you do it, please use the mysqlbug script so that we can get information about your system and MySQL version. See the section called “How to Report Bugs or Problems”.
Make sure to include the output of mysql-test-run, as well as contents of all .reject files in mysql-test/r directory.
If a test in the test suite fails, check whether the test fails also when run by its own:
cd mysql-test mysql-test-run --local test-name
If this fails, then you should configure MySQL with --with-debug and run mysql-test-run with the --debug option. If this also fails send the trace file var/tmp/master.trace to ftp://ftp.mysql.com/pub/mysql/upload/ so that we can examine it. Please remember to also include a full description of your system, the version of the mysqld binary and how you compiled it.
Try also to run mysql-test-run with the --force option to see whether there is any other test that fails.
If you have compiled MySQL yourself, check our manual for how to compile MySQL on your platform or, preferable, use one of the binaries we have compiled for you at http://dev.mysql.com/downloads/. All our standard binaries should pass the test suite !
If you get an error, like Result length mismatch or Result content mismatch it means that the output of the test didn't match exactly the expected output. This could be a bug in MySQL or that your mysqld version produces slight different results under some circumstances.
Failed test results are put in a file with the same base name as the result file with the .reject extension. If your test case is failing, you should do a diff on the two files. If you cannot see how they are different, examine both with od -c and also check their lengths.
If a test fails totally, you should check the logs file in the mysql-test/var/log directory for hints of what went wrong.
If you have compiled MySQL with debugging you can try to debug this by running mysql-test-run with the --gdb and/or --debug options. See the section called “Creating Trace Files”.
If you have not compiled MySQL for debugging you should probably do that. Just specify the --with-debug options to configure! See the section called “MySQL Installation Using a Source Distribution”.