There are four processes that are important to know about when using MySQL Cluster. We will cover how to work with those processes, which options to use when starting and so forth.
mysqld is the traditional MySQL server process. To be used with MySQL Cluster it needs to be built with support for the NDB Cluster storage engine. If the mysqld binary has been built in such a manner, the NDB Cluster storage engine is still disabled by default.
To enable the NDB Cluster storage engine there are two ways. Either use --ndbcluster as a startup option when starting mysqld or insert a line with ndbcluster in the [mysqld] section of your my.cnf file. An easy way to verify that your server runs with support for the NDB Cluster storage engine is to issue the command SHOW ENGINES from a mysql client. You should see YES for the row listing NDBCLUSTER. If you see NO, you are not running a mysqld that is compiled with NDB Cluster support enabled. If you see DISABLED, then you need to enable it in the my.cnf configuration file.
The MySQL server needs to know how to get the configuration of the cluster. To access this configuration, it needs to know three things:
Its own node ID in the cluster.
The hostname (or IP address) where the management server resides.
The port on which it can connect to the management server.
The node ID can be skipped from MySQL 4.1.5 on, because a node ID can be dynamically allocated.
There are currently three possible ways to provide this information to the mysqld process. The recommended option is to set the connect string as a mysqld parameter ndb-connectstring either when starting mysqld or in my.cnf. You may alternatively include this information in a file called Ndb.cfg. This file should reside in the data directory of the MySQL Server. As another option you set an environment variable called NDB_CONNECTSTRING. The string is the same in all cases, "[nodeid=<id>;][host=]<host>:<port>". If no information is supplied then it will default to "host=localhost:2200".
shell> mysqld --ndb-connectstring=ndb_mgmd.mysql.com:2200
ndb_mgmd.mysql.com is the host where the management server resides, and it is listening to port 2200.
With this setup the MySQL server will be a full citizen of MySQL Cluster and will be fully aware of all storage nodes in the cluster and their status. It will setup connection to all storage nodes and will be able to use any storage node as a transaction coordinator and to access their data for reading and updating.
ndbd is the process that is used to handle all the data in the tables using the NDB Cluster storage engine. This is the process that contains all the logic of distributed transaction handling, node recovery, checkpointing to disk, online backup, and lots of other functionality.
In a cluster there is a set of ndbd processes cooperating in handling the data. These processes can execute on the same computer or on different computers, in a completely configurable manner.
Before MySQL 4.1.5, ndbd process should start from a separate directory. The reason for this was that ndbd generates a set of log files in its starting directory.
In MySQL 4.1.5, this was changed such that the files are placed in the directory specified by DataDir in the configuration file. Thus ndbd can be started from anywhere.
These log files are (the 2 is the node ID):
ndb_2_error.log (was error.log in version 4.1.3) is a file that contains information of all the crashes which the ndbd process has encountered and a smaller error string and reference to a trace file for this crash. An entry could like this:
Date/Time: Saturday 31 January 2004 - 00:20:01 Type of error: error Message: Internal program error (failed ndbrequire) Fault ID: 2341 Problem data: DbtupFixAlloc.cpp Object of reference: DBTUP (Line: 173) ProgramName: NDB Kernel ProcessID: 14909 TraceFile: ndb_2_trace.log.2 ***EOM***
ndb_2_trace.log.1 (was NDB_TraceFile_1.trace in version 4.1.3) is a trace file describing exactly what happened before the error occurred. This information is useful for the MySQL Cluster team when analysing any bugs occurring in MySQL Cluster. The information in this file will be described in the section MySQL Cluster Troubleshooting. There can be a configurable number of those trace files in the directory before old files are overwritten. 1 in this context is simply the number of the trace file.
ndb_2_trace.log.next (was NextTraceFileNo.log in version 4.1.3) is the file that keeps track of what the next trace file number is to be.
ndb_2_out.log is the file which contains any data printed by the ndbd process. 2 in this context is the node ID. (this file only applies if ndbd is started in dameon mode which is default from 4.1.5, was node2.out in version 4.1.3)
ndb_2.pid is a file containing the process id of the ndbd when started as a daemon (default from 4.1.5 and named node2.pid in version 4.1.3). It also functions as a lock file to avoid the starting of nodes with the same id.
ndb_2_signal.log (was Signal.log in version 4.1.3) is a file which is only used in debug versions of ndbd where it is possible to trace all incoming, outgoing and internal messages with their data in the ndbd process.
It is recommended to not use a directory mounted through NFS because in some environments that can cause problems with the lock on the pid-file remaining even after the process has stopped.
Also when starting the ndbd process it may be necessary to specify the hostname of the management server and the port it is listening to, optionally one may specify node ID the process is to use. Again there are three ways of specifying this information. Either in a string in the file Ndb.cfg, this file should be stored in the starting directory of the ndbd process. The second option is to set the environment variable NDB_CONNECTSTRING before starting the process. The third option is to use a command line parameter as in the example below. See previous section for connect string format.
shell> ndbd --connect-string="nodeid=2;host=ndb_mgmd.mysql.com:2200"
When ndbd starts it will actually start two processes. The starting process is called the "angel" and its only job is to discover when the execution process has completed, and then restart the ndbd process if configured to do so. Thus if one attempts to kill ndbd through the kill command in Unix, it is necessary to kill both processes. A more proper way to handle the stopping of ndbd processes is to use the management client and stop the process from there.
The execution process uses one thread for all activities in reading, writing, and scanning data and all other activities. This thread is designed with asynchronous programming so it can easily handle thousands of concurrent activites. In addition there is a watch-dog thread supervising the execution thread to ensure it doesn't stop in an eternal loop or other problem. There is a pool of threads handling file I/O. Each thread can handle one open file. In addition threads can be used for connection activities of the transporters in the ndbd process. Thus in a system that performs a large number of activities including update activities the ndbd process will consume up to about 2 CPUs if allowed to. Thus in a large SMP box with many CPUs it is recommended to use several ndbd processes which are configured to be part of different node groups.
The management server is the process that reads the configuration file of the cluster and distributes this information to all nodes in the cluster requesting it. It also maintains the log of cluster activities. Management clients can connect to the management server and use commands to check status of the cluster in various aspects.
As of MySQL 4.1.5, it is no longer to specify a connect string when starting the management server. If however you are using several management servers a connect string should be provided and all nodes in the cluster should specify their nodeid's explicitly.
The following files are created or used by ndb_mgmd in its starting directory of ndb_mgmd. From MySQL 4.1.5, the log and PID files will be placed in the DataDir specified in the configuration file:
config.ini is the configuration file of the cluster. This is created by the user and read by the management server. How to write this file is described in the section MySQL Cluster Configuration.
ndb_1_cluster.log (was cluster.log in version 4.1.3) is the file where events in the cluster are reported. Examples of events are checkpoints started and completed, node failures and nodes starting, levels of memory usage passed and so forth. The events reported are described in the section the section called “Management of MySQL Cluster”.
ndb_1_out.log (was node1.out in version 4.1.3) is the file used for stdout and stderr when executing the management server as a daemon process. 1 in this context is the node ID.
ndb_1.pid (was node1.pid in version 4.1.3) is the PID file used when executing the management server as a daemon process. 1 in this context is the node ID.
ndb_1_cluster.log.1 (was cluster.log.1 in version 4.1.3) when the cluster log becomes bigger than one million bytes then the file is renamed to this name where 1 is the number of the cluster log file, so if 1, 2, and 3 already exists the next will be having the number 4 instead.
The final important process to know about is the management client. This process is not needed to run the cluster. Its value lies in its ability to check status of the cluster, start backups, and perform other management activities. It does so by providing access to a set of commands.
Actually the management client is using a C API that is used to access the management server so for advanced users it is also possible to program dedicated management processes which can do similar things as the management client can do.
When starting the management client, it is necessary to state the hostname and port of the management server as in the example below. The default is localhost as host and port number 2200.
shell> ndb_mgm localhost 2200
If the binary includes support for the NDB Cluster storage engine the default disabling of support for the NDB storage engine can be overruled by using this option. Using the NDB Cluster storage engine is necessary for using MySQL Cluster.
Disable the NDB Cluster storage engine. This is disabled by default for binaries where it is included. So this option only applies if the server was configured to use the NDB Cluster storage engine.
When using the NDB storage engine, it is possible to point out the management server that distributes the cluster configuration by setting the connect string option.
These options only starts the program to print its command options.
For ndbd it is also possible to set the connect string to the management server as a command option.
shell> ndbd --connect-string="nodeid=2;host=ndb_mgmd.mysql.com:2200"
Instructs ndbd to execute as a daemon process. From MySQL 4.1.5 on, this is the default behavior.
Instructs ndbd not to start as a daemon process. Useful when ndbd is debugged and one wants printouts on the screen.
Instructs ndbd to perform an initial start. An initial start will erase any files created by earlier ndbd instances for recovery. It will also recreate recovery log files which on some operating systems can take a substantial amount of time.
An initial start is only to be used at the very first start of the ndbd process. It removes all files from the file system and creates all REDO log files. When performing a software upgrade which has changed the file contents on any files it is also necessary to use this option when restarting the node with a new software version of ndbd. Finally it could also be used as a final resort if for some reason the node restart or system restart doesn't work. In this case be aware that destroying the contents of the file system means that this node can no longer be used to restore data.
This option does not affect any backup files created.
The previous possibility to use -i for this option was removed to ensure that this option is not used by mistake.
Instructs ndbd not to automatically start. ndbd will connect to the management server and get the configuration and initialise communication objects. It will not start the execution engine until requested to do so by the management server. The management server can request by command issued by the management client.
Prints the version number of the ndbd process. The version number is the MySQL Cluster version number. It is important because at startup the MySQL Cluster processes verifies that the versions of the processes in the cluster can co-exist in the cluster. It is also important for online software upgrade of MySQL Cluster (see section Software Upgrade of MySQL Cluster).
This can be used only for versions compiled with debug information. It is used to enable printouts from debug calls in the same manner as for the mysqld process.
Prints a short description of the available command options.
These options only start the program to print its command options.
Instructs the management server which file to use as configuration file. This option must be specified. The file name defaults to config.ini.
Instructs ndb_mgmd to start as a daemon process. This is the default behavior.
Instructs the management server not to start as a daemon process.
Prints the version number of the management server. The version number is the MySQL Cluster version number. The management server can check that only versions capable of working with its versions are accepted and provided with the configuration information.
This can only be used for versions compiled with debug information. It is used to enable printouts from debug calls in the same manner as for the mysqld process.
These options only starts the program to print its command options.
To start the management client it is necessary to specify where the management server resides. This means specifying the hostname and the port. The default hostname is localhost and the default port is 2200.
If the connection to the management server is broken it is possible to perform only a specified amount of retries before reporting a fault code to the user. The default is that it keeps retrying every 5 seconds until it succeeds.