Go to the first, previous, next, last section, table of contents.


17 MySQL Cluster

MySQL Cluster uses the new NDBCluster storage engine to enable running several MySQL servers in a cluster. The NDBCluster storage engine is available from MySQL source release 4.1.2, and in binary releases from MySQL-Max 4.1.3.

Supported operating systems are currently Linux, Mac OS X, and Solaris. Naturally, we are working to make NDBCluster run on all of the operating systems that MySQL itself also runs on, including Windows.

This chapter represents work in progress. Other documents describing MySQL Cluster can be found at http://www.mysql.com/cluster/ and http://dev.mysql.com/doc/#cluster.

You may also wish to subscribe to the MySQL Cluster mailing list. See http://lists.mysql.com/.

17.1 MySQL Cluster Overview

MySQL Cluster is a new technology to enable clustering of in-memory databases in a share-nothing system. The share-nothing architecture allows the system to work with very inexpensive hardware, without any specific requirement on hardware or software. It also does not have any single point of failure since each component has its own memory and disk.

MySQL Cluster is an integration of the standard MySQL server with an in-memory clustered storage engine, called NDB. In our documentation, the term NDB refers to the storage engine specific part of the setup, whereas MySQL Cluster refers to the combination of MySQL and the new storage engine.

A MySQL Cluster consists of computers with a set of processes executing several MySQL servers, storage nodes for NDB Cluster, management servers and possibly also specialized data access programs. All these programs work together to form MySQL Cluster. When data is stored in the NDBCluster storage engine, the tables are stored in the storage nodes for NDB Cluster. Those tables are directly accessible also from all other MySQL servers in the cluster. Thus, if one application updates the salary of an employee all other MySQL servers that query this data can see it immediately.

The data stored in the storage nodes for MySQL Cluster can be mirrored and can handle failures of storage nodes with no other impact than that a number of transactions are aborted due to losing the transaction state. This should cause no problems bcause transactional applications should be written to handle transaction failure.

By bringing MySQL Cluster to the open source world, MySQL makes clustered data management with high availability, high performance and scalability available to all who need it.

17.2 Basic MySQL Cluster Concepts

NDB is an in-memory storage engine offering high-availability and data-persistence features. NDB can (although this requires extensive knowledge) be used as an independent database system, supporting the traditional relational data model with full ACID transactions.

The NDB storage engine can be configured with a range of fail-over and load-balancing options, but it is easiest to start with the storage engine at the cluster level. The NDB storage engine of MySQL Cluster contains a complete set of data, dependent only on other data within the cluster itself.

A MySQL Cluster may also replicate clustered data to other MySQL Clusters, but this is a complex configuration. Here, we will focus on how to set up a single MySQL Cluster consisting of an NDB storage engine and some MySQL servers.

The cluster part of MySQL Cluster is currently configured independently from the MySQL servers. In an MySQL Cluster each part of the cluster is considered to be a node.

Note: A node in many contexts is often a computer, but for MySQL Cluster it is a process. There can be any number of nodes on a single computer.

Each node has a type, and there can be multiple nodes in the MySQL Cluster of each type. In a minimal MySQL Cluster configuration, there will be at least three nodes:

We refer to these cluster processes as nodes in the cluster. Setting up the configuration of the cluster involves configuring each individual node in the cluster and setting up each individual communication link between the nodes in the cluster. MySQL Cluster currently is designed with the intention that storage nodes are homogenous in terms of processor power, memory space, and communication bandwidth. Also, to enable one point of configuration, it was decided to move the entire cluster configuration to one configuration file.

The management server manages the cluster configuration file and the cluster log. All nodes in the cluster contact the management server to retrieve their part of the configuration, so they need a way to determine where the management server resides. When interesting events occur in the storage nodes, they transfer the information of these events to the management server, which then writes the information to the cluster log.

In addition, there are any number of clients to the cluster. These are of two types. First, there are the normal MySQL clients that are no different for MySQL Cluster. MySQL Cluster can be accessed from all MySQL applications written in PHP, Perl, C, C++, Java, Ruby, and so forth. Second, there are management clients. These clients access the management server and provide commands to start and stop nodes gracefully, to start and stop message tracing (only in debug versions), to print configuration, to show node status of all nodes in the cluster, to show versions of all nodes in the cluster, to start and stop backups, and so forth.

17.3 MySQL Cluster Configuration

A MySQL server that is part of MySQL Cluster differs in only one aspect from what we are used to, it has an additional storage engine (NDB or NDBCLUSTER), which is initially disabled.

Except for this, the MySQL server is not much different than what we are used to from previous MySQL releases, except any other new 4.1 features, of course. By default, the MySQL is configured with the NDB storage engine disabled (to not needlessly allocate resources); to enable it you need to modify `my.cnf'.

Also, as the MySQL daemon is an API client to the NDB storage engine, the minimal configuration data needed to access the MGM node from the MySQL server must be set. When this is done, then all MGM nodes (one is sufficient to start) and DB nodes must be up and running before starting the MySQL server.

17.3.1 Building from Source Code

NDBCluster is available in binary distributions from MySQL-Max 4.1.3.

If you choose to build from source terball or the BitKeeper MySQL 4.1 tree, make sure you add the --with-ndbcluster option when running `configure'. You could also simply use the BUILD/compile-pentium-max build script. This script also includes OpenSSL, so you either have to get OpenSSL of modify the build script to exclude it.

Apart from these things, you can just follow the standard instructions to build your own binaries, run the tests and perform the installation procedure. See section 2.3.3 Installing from the Development Source Tree.

17.3.2 Installing the Software

You need to have all the MGM and DB nodes up and running first, and this will probably be the most time-consuming part of the configuration, if for no other reason than because we will assume that you are already familiar with MySQL to a certain extent. As for the MySQL configuration and the `my.cnf' file, this is very straightforward, and this section only covers the differences from configuring MySQL without clustering.

17.3.3 Quick Test Setup of MySQL Cluster

This section describes how to quickly configure and start the simplest possible MySQL Cluster setup. This will make you familiar with the basic concepts. Then, read the other sections and design your desired setup.

A couple of directories need to be created (as root):

shell> mkdir /var/lib/mysql-cluster
shell> mkdir /var/lib/mysql-cluster/node1
shell> mkdir /var/lib/mysql-cluster/node2

In the directory of node1, create a file `config.ini' with the following contents:

# file "config.ini" - showing minimal setup with 1 DB node
# This file is placed in the start directory of ndb_mgmd,
# the management server.

[COMPUTER DEFAULT]

[DB DEFAULT]
NoOfReplicas: 1

[API DEFAULT]

[MGM DEFAULT]
ArbitrationRank: 2

[TCP DEFAULT]
PortNumber: 28002

[COMPUTER]
Id:1
HostName: localhost

[MGM]
Id:1
ExecuteOnComputer: 1
PortNumber: 2200

[DB]
Id: 2
ExecuteOnComputer: 1
FileSystemPath: /var/lib/mysql-cluster/node2/

[API]
Id: 3
ExecuteOnComputer: 1

In the same directory (`node1'), also create a file `Ndb.cfg' (note the capital N) containing the following information:

nodeid=1;host=localhost:2200

You can now start the management server. Do this as follows:

shell> cd /var/lib/mysql-cluster/node1
shell> ndb_mgmd -c config.ini -d

In the `node2' directory, also create a file `Ndb.cfg' but with this information:

nodeid=2;host=localhost:2200

To start the single DB node in this simple setup:

shell> cd /var/lib/mysql-cluster/node2
shell> ndbd -d

Please note that if you installed from the binary tarball, you will need to explicitly specify the path of the @command{ndb_mgmd} and @command{ndbd} servers. They would be in the `/usr/local/mysql/bin' directory.

Finally, go to the MySQL data directory (`/var/lib/mysql' or `/usr/local/mysql/data' and ensure that the `my.cnf' file contains the option necessary to enable the NDBCluster storage engine:

[mysqld]
ndbcluster

Create a file `Ndb.cfg' here (i.e., in the MySQL data directory) with this information:

nodeid=3;host=localhost:2200

You can now start the MySQL server as usual:

shell> mysqld_safe --user=mysql &

Wait a moment to make sure the MySQL server is running properly. If you see a notice ``mysql ended'', you need to check the server's `.err' file to find out what went wrong.

If all went well so far, we can now start using the cluster:

shell> mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.3-beta-max-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> SHOW STORAGE ENGINES;
+------------+---------+-------------------------------------------...
| Engine     | Support | Comment                                   ...
+------------+---------+-------------------------------------------...
...
| NDBCLUSTER | YES     | Clustered, fault tolerant memory based tables
| NDB        | YES     | Alias for NDBCLUSTER                      ...
...

mysql> USE test;
Database changed

mysql> CREATE TABLE ctest (i INT) ENGINE=NDBCLUSTER;
Query OK, 0 rows affected (0.59 sec)

mysql> SHOW CREATE TABLE ctest \G
*************************** 1. row ***************************
       Table: ctest
Create Table: CREATE TABLE `ctest` (
  `i` int(11) default NULL
) ENGINE=ndbcluster DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

As you will have noticed, there are still some aspects of the initial configuration that can be simplified and otherwise improved, and we are working on this. However, it may already pass the famous ``15 minute test'' for being able to install a working setup. Clustering is yours. Enjoy.

17.3.4 Configuration File

The configuration of MySQL Cluster is contained in a configuration file read by the management server and distributed to all processes involved in the cluster. This file contains a description of all involved nodes in the cluster, configuration parameters for the storage nodes and configuration parameters for all connections between the nodes in the cluster.

17.3.4.1 An example configuration in a MySQL Cluster

Currently the configuration file in INI format, and is named `config.ini' It is placed in the start directory of ndb_mgmd (the management server).

Preset defaults are available for most parameters, and defaults can be specified in the `config.ini' file itself. For example, if all DB nodes use the same size of data memory, which is not the same as the preset default size, then a DB DEFAULT entry can be created that specifies the default data memory size for all DB node entries.

The INI format consists of sections preceded by section headings (surrounded by square brackets), followed by settings and values. One deviation from the standard format being that the setting name and value are separated by a colon (`:') instead of an equal sign (`='), and another is that the sections aren't unique. Instead, unique entries (such as two nodes of the same type) are identified by a unique ID.

A minimalistic configuration file needs to define the computers involved in the cluster and which nodes are involved in the cluster and what computers these nodes are placed on.

An example of a still fairly minimalistic configuration file for a cluster with one management server, two storage nodes and two MySQL servers is shown below:

# file "config.ini" - 2 DB nodes and 2 mysqld
# This file is placed in the start directory of ndb_mgmd,
# the management server.

[COMPUTER DEFAULT]

[DB DEFAULT]
NoOfReplicas: 2

[API DEFAULT]

[MGM DEFAULT]
ArbitrationRank: 2

[TCP DEFAULT]
PortNumber: 28002

[COMPUTER]
Id: 1
Hostname: ndb_mgmd.mysql.com

[COMPUTER]
Id: 2
HostName: ndbd_2.mysql.com

[COMPUTER]
Id: 3
HostName: ndbd_3.mysql.com

[COMPUTER]
Id: 4
HostName: mysqld_4.mysql.com

[COMPUTER]
Id: 5
HostName: mysqld_5.mysql.com

[MGM]
Id: 1
ExecuteOnComputer: 1
PortNumber: 2200

[DB]
Id: 2
ExecuteOnComputer: 2
FileSystemPath: /var/lib/mysql-cluster/node2/

[DB]
Id: 3
ExecuteOnComputer: 3
FileSystemPath: /var/lib/mysql-cluster/node3/

[API]
Id: 4
ExecuteOnComputer: 4

[API]
Id: 5
ExecuteOnComputer: 5

#[TCP]
#NodeId1: 2
#NodeId2: 3
#SendBufferMemory: 512K

There are six different sections in the config file. COMPUTER defines the computers in the cluster. DB defines the storage nodes in the cluster. API defines the MySQL server nodes in the cluster. MGM defines the management server node in the cluster. TCP defines TCP/IP connections between nodes in the cluster, TCP/IP is the default connection mechanism between two nodes. SHM defines shared-memory connections between nodes. This is only available in binaries which have been built with the flag --with-ndb-shm.

For each section one can define DEFAULTs. All parameters are currently case sensitive.

17.3.4.2 Defining the computers in a MySQL Cluster

The COMPUTER section has no real significance other than serving as a way to avoid the need of defining host names for each node in the system. All parameters mentioned here are mandatory.

[COMPUTER]Id
This is an internal identity in the configuration file. Later on in the file one refers to this computer by the id. It is an integer.
[COMPUTER]HostName
This is the host name of the computer. It is also possible to use an IP address rather than the host name.

17.3.4.3 Defining the management server in a MySQL Cluster

The MGM section is used to configure the behaviour of the management server in various aspects. The mandatory parameters are Id, ExecuteOnComputer and PortNumber. All other parameters can be left out and will in that case receive the default value.

[MGM]Id
This identity is the node id used as the address of the node in all cluster internal messages. This is an integer between 1 and 63. Each node in the cluster has a unique identity.
[MGM]ExecuteOnComputer
This is referring to one of the computers defined in the computer section.
[MGM]PortNumber
This is the port number which the management server will listen on for configuration requests and management commands.
[MGM]LogDestination
This parameter specifies where to send the cluster log. There are three possible places to send it to and they can be sent to all three in parallel if desired. These three are CONSOLE, SYSLOG and FILE. One assigns all of these three in one string with each part separated by a ';'. CONSOLE means putting it to stdout, no more parameters are needed.
CONSOLE
SYSLOG means sending it to a syslog facility. It is necessary to specify the facility for this parameter. The poossible facilities are auth, authpriv, cron, daemon, ftp, kern, lpr, mail, news, syslog, user, uucp, local0, local1,local2, local3, local4, local5, local6, local7. Note that every facility is not necessarily supported by every operating system.
SYSLOG:facility=syslog
FILE means sending the cluster log to a regular file on the machine. It is necessary to specify the name of this file, the maximum size of the file until a new file is opened and the old is renamed with filename extended by .x where x is the next number not used yet on the file. It is also necessary to specify maximum number of rolled files.
FILE:filename=cluster.log,maxsize=1000000,maxfiles=6
Multiple log destinations can be specified as in the following example.
CONSOLE;SYSLOG:facility=local0;FILE:filename=/var/log/mgmd
The default of this parameter is FILE:filename=cluster.log,maxsize=1000000,maxfiles=6.
[MGM]ArbitrationRank
This parameter is used to define which nodes can act as an arbitrator. MGM nodes and API nodes can be arbitrators. 0 means it isn't used as arbitrator, 1 high priority and 2 low priority. A normal configuration uses the management server as arbitrator setting the ArbitrationRank to 1 (which is the default) and setting all APIs to 0 (not default in MySQL 4.1.3).
[MGM]ArbitrationDelay
If setting this to anything else than 0 it means that the management server will delay responses to the arbitration requests. Default is no delay and this should not be necessary to change.

17.3.4.4 Defining the storage nodes in a MySQL Cluster

The DB section is used to configure the behaviour of the storage nodes. There are many parameters specified that controls the buffer sizes, pool sizes, time-out parameters and so forth. The only mandatory parameters are the Id, ExecuteOnComputer and NoOfReplicas. Most parameters should be set in the DB DEFAULT section. Only parameters explicitly stated as possible to have local values are allowed to be changed in the DB section. Id and ExecuteOnComputer needs to be defined in the local DB section.

The first parameters to define are all mandatory, everyone except for NoOfReplicas should be defined per storage node.

For each parameter it is possible to use k, M and G which are then converted to 1024, (1024*1024) and (1024* 1024*1024). So 100k means 102400. Parameters and values are currently case sensitive.

The max values, min values and default values of the parameters are currently worked upon for best experience for first-time users. Also some unnecessary low max values will be changed soon.

[DB]Id
This identity is the node id used as the address of the node in all cluster internal messages. This is an integer between 1 and 63. Each node in the cluster has a unique identity.
[DB]ExecuteOnComputer
This is referring to one of the computers defined in the computer section.
[DB]NoOfReplicas
This parameter can only be set in the DB DEFAULT section since it is a global parameter. It defines the number of replicas for each table stored in the cluster. This parameter also specifies the size of node groups. A node group is a set of nodes that all store the same information. Node groups are formed implicitly. The first node group is formed by the storage nodes with the lowest node identities. And the next by the next lowest node identities. As an example presume we have 4 storage nodes and NoOfReplicas is set to 2. The four storage nodes have node id 2, 3, 4 and 5. Then the first node group will be formed by node 2 and node 3. The second node group will be formed by node 4 and node 5. It is important to configure the cluster in such a manner such that nodes in the same node groups are not placed on the same computer. This would cause a single HW failure to cause a cluster crash. There is no default value and the maximum number is 4.
[DB]FileSystemPath
This parameter has no default value, it defines the file directory which is used by this storage node. The directory must be created before starting the ndbd process. If you use the recommended directory hierarchy, you will first need to create the `/var/lib/mysql-cluster' directory, and then a subdirectory for each DB node running on this computer. We suggest you simple name these node1, node2, etc.

DataMemory and IndexMemory are the parameters that specify the size of memory segments used to store the actual records and their indexes. It is important to understand how DataMemory and IndexMemory is used to understand how to set these parameters which for most user cases will need to be updated to reflect the usage of the cluster.

[DB]DataMemory
This parameter is one of the most important parameters since it defines the space available to store the actual records in the database. The entire DataMemory will be allocated in memory so it is important that the machine contains enough memory to handle the DataMemory size. The DataMemory is used to store two things. It stores the actual records. Each record is currently of fixed size. So VARCHAR fields are stored as fixed size fields. There is an overhead on each record on 16 bytes normally. Additionally each record is stored in a 32 kByte page with 128 byte page overhead. There will also be a small amount of waste for each page since records are only stored in one page. The maximum record size for the fields is currently 8052 bytes. The DataMemory is also used to store ordered indexes. Ordered indexes uses about 10 bytes per record. Each record in the table is always represented in the ordered index. The DataMemory consists of 32kByte pages. These pages are allocated to partitions of the tables. Each table is normally partitioned with the same number of partitions as there are storage nodes in the cluster. Thus for each node there are the same amount of partitions (=fragments) as the NoOfReplicas is set to. Once a page has been allocated to a partition it is currently not possible to bring it back to the pool of free pages. The methods to restore pages to the pool is by deleting the table. Performing a node recovery also will compress the partition since all records are inserted into an empty partition from another live node. Another important aspect is that the DataMemory also contains UNDO information for records. For each update of a record a copy record is allocated in the DataMemory. Also each copy record will also have an instance in the ordered indexes of the table. Unique hash indexes are only updated when the unique index fields are updated and in that case a new entry in the index table is inserted and at commit the old entry is deleted. Thus it is necessary to also allocate memory to be able to handle the largest transactions which are performed in the cluster. Performing large transactions has no advantage in MySQL Cluster other than the consistency of using transactions which is the whole idea of transactions. It is not faster and consumes large amounts of memory. The default DataMemory size is 80000 kBytes. The minimum size is 1 MByte and the maximum size is slightly more than 3 GBytes. In reality the maximum size has to be adapted so that the process doesn't start swapping when using the maximum size of the memory.
[DB]IndexMemory
The IndexMemory is the parameter that controls the amount of storage used for hash indexes in MySQL Cluster. Hash indexes are always used for primary key indexes and unique indexes and unique constraints. Actually when defining a primary key and a unique index there will be two indexes created in MySQL Cluster. One index is a hash index which is used for all tuple accesses and also for lock handling. It is also used to ensure unique constraints. The size of the hash index is 25 bytes plus the size of the primary key. For primary keys larger than 32 bytes another 8 bytes is added for some internal references. Thus for a table defined as
CREATE TABLE example (a int not null, b int not null, c int not null,
                      PRIMARY KEY(a), UNIQUE(b)) ENGINE=NDBCLUSTER;
We will have 12 bytes overhead (no nullable fields saves 4 bytes of overhead) plus 12 bytes of data per record. In addition we will have two ordered indexes on a and b consuming about 10 bytes each per record. We will also have a primary key hash index in the base table with roughly 29 bytes per record. The unique constraint is implemented by a separate table with b as primary key and a as a field. This table will consume another 29 bytes of index memory per record in the table and also 12 bytes of overhead plus 8 bytes of data in the record part. Thus for 1 million records we will need 58 MBytes of index memory to handle the hash indexes for the primary key and the unique constraint. For the DataMemory part we will need 64 MByte of memory to handle the records of the base table and the unique index table plus the two ordered index tables. The conclusion is that hash indexes takes up a fair amount of memory space but in return they provide very fast access to the data. They are also used in MySQL Cluster to handle uniqueness constraints. Currently the only partitioning algorithm is hashing and the ordered indexes are local to each node and can thus not be used to handle uniqueness constraints in the general case. An important point for both IndexMemory and DataMemory is that the total database size is the the sum of all DataMemory and IndexMemory in each node group. Each node group is used to store replicated information so if there are four nodes with 2 replicas there will be two node groups and thus the total DataMemory available is 2*DataMemory in each of the nodes. Another important point is about changes of DataMemory and IndexMemory. First of all it is highly recommended to have the same amount of DataMemory and IndexMemory in all nodes. Since data is distributed evenly over all nodes in the cluster the size available is no better than the smallest sized node in the cluster times the number of node groups. DataMemory and IndexMemory can be changed, it is dangerous to decrease it since that can easily lead to a node that will not be able to restart or even a cluster not being able to restart since there is not enough memory space for the tables needed to restore into the starting node. Increasing it should be quite ok, but it is recommended that such upgrades are performed in the same manner as a software upgrade where first the configuration file is updated, then the management server is restarted and then one storage node at a time is restarted by command. More IndexMemory is not used due to updates but inserts are inserted immediately and deletes are not deleted until the transaction is committed. The default IndexMemory size is 24000 kBytes. The minimum size is 1 MByte and the maximum size is around 1.5 GByte.

The next two parameters are important since they will affect the number of parallel transactions and the sizes of transactions that can be handled by the system. MaxNoOfConcurrentTransactions sets the number of parallel transactions possible in a node and MaxNoOfConcurrentOperations sets the number of records that can be in update phase or locked simultaneously.

Both of these parameters and particularly MaxNoOfConcurrentOperations are likely targets for users setting specific values and not using the default value. The default value is set for systems using small transactions and to ensure not using too much memory in the default case.

[DB]MaxNoOfConcurrentTransactions
For each active transaction in the cluster there needs to be also a transaction record in one of the nodes in the cluster. The role of transaction coordination is spread among the nodes and thus the total number of transactions records in the cluster is the amount in one times the number of nodes in the cluster. Actually transaction records are allocated to MySQL servers, normally there is at least one transaction record allocated in the cluster per connection that uses or have used a table in the cluster. Thus one should ensure that there is more transaction records in the cluster than there are concurrent connections to all MySQL servers in the cluster. This parameter has to be the same in all nodes in the cluster. Changing this parameter is never safe and can cause a cluster crash. When a node crashes one of the node (actually the oldest surviving node) will build up the transaction state of all transactions ongoing in the crashed node at the time of the crash. It is thus important that this node has as many transaction records as the failed node. The default value for this parameter is 4096.
[DB]MaxNoOfConcurrentOperations
This parameter is likely to be subject for change by users. Users only performing short, small transactions don't need to set this parameter very high. Applications desiring to be able to perform rather large transactions involving many records need to set this parameter higher. For each transaction that updates data in the cluster it is required to have operation records. There are operation records both in the transaction coordinator and in the nodes where the actual updates are performed. The operation records contain state information needed to be able to find UNDO records for rollback, lock queues and much other state information. To dimension the cluster to handle transactions where 1 million records are updated simultaneously one should set this parameter to 1 million divided by the number of nodes. Thus for a cluster with four storage nodes one should set this parameter to 250000. Also read queries which set locks use up operation records. Some extra space is allocated in the local nodes to cater for cases where the distribution is not perfect over the nodes. When queries translate into using the unique hash index there will actually be two operation records used per record in the transaction. The first one represents the read in the index table and the second handles the operation on the base table. The default value for this parameter is 8192. The maximum is 1000000.

The next set of parameters are used for temporary storage in the midst of executing a part of a query in the cluster. All of these records will have been released when the query part is completed and is waiting for the commit or rollback.

Most of the defaults for these parameters will be ok for most users. Some high-end users might want to increase those to enable more parallelism in the system and some low-end users might want to decrease them to save memory.

[DB]MaxNoOfConcurrentIndexOperations
For queries using a unique hash index another set of operation records are temporarily used in the execution phase of the query. This parameter sets the size of this pool. Thus this record is only allocated while executing a part of a query, as soon as this part has been executed the record is released. The state needed to handle aborts and commits is handled by the normal operation records where the pool size is set by the parameter MaxNoOfConcurrentOperations. The default value of this parameter is 8192. Only in rare cases of extremely high parallelism using unique hash indexes should this parameter be necessary to increase. To decrease could be performed for memory savings if the DBA is certain that such high parallelism is not occurring in the cluster.
[DB]MaxNoOfFiredTriggers
The default value of MaxNoOfFiredTriggers is 1000. Normally this value should be sufficient for most systems. In some cases it could be decreased if the DBA feels certain the parallelism in the cluster is not so high. This record is used when an operation is performed that affects a unique hash index. Updating a field which is part of a unique hash index or inserting/deleting a record in a table with unique hash indexes will fire an insert or delete in the index table. This record is used to represent this index table operation while its waiting for the original operation that fired it to complete. Thus it is short lived but can still need a fair amount of records in its pool for temporary situations with many parallel write operations on a base table containing a set of unique hash indexes.
[DB]TransactionBufferMemory
This parameter is also used for keeping fired operations to update index tables. This part keeps the key info and field information for the fired operations. It should be very rare that this parameter needs to be updated. Also normal read and write operations use a similar buffer. This buffer is even more short term in its usage so this is a compile time parameter set to 4000*128 bytes, thus 500 kBytes. The parameter is ZATTRBUF_FILESIZE in Dbtc.hpp. A similar buffer for key info exists which contains 4000*16 bytes, 62.5 kBytes of buffer space. The parameter in this case is ZDATABUF_FILESIZE in Dbtc.hpp. Dbtc is the module for handling the transaction coordination. Similar parameters exist in the module Dblqh taking care of the reads and updates where the data is located. In Dblqh.hpp with ZATTRINBUF_FILESIZE set to 10000*128 bytes, 1250 kBytes and ZDATABUF_FILE_SIZE, set to 10000*16 bytes, roughly 156 kBytes of buffer space. No known instances of that any of those compile time limits haven't been big enough has been reported so far or discovered by any of our extensive test suites. The default size of the TransactionBufferMemory is 1000 kBytes.
[DB]MaxNoOfConcurrentScans
This parameter is used to control the amount of parallel scans that can be performed in the cluster. Each transaction coordinator can handle the amount of parallel scans defined by this parameter. Each scan query is performed by scanning all partitions in parallel. Each partition scan will use a scan record in the node where the partition is located. The number of those records is the size of this parameter times the number of nodes so that the cluster should be able to sustain maximum number of scans in parallel from all nodes in the cluster. Scans are performed in two cases. The first case is when no hash or ordered indexes exists to handle the query. In this case the query is executed by performing a full table scan. The second case is when there is no hash index to support the query but there is an ordered index. Using the ordered index means executing a parallel range scan. Since the order is only kept on the local partitions it is necessary to perform the index scan on all partitions. The default value of MaxNoOfConcurrentScans is 25. The maximum value is 500.
[DB]NoOfFragmentLogFiles
This parameter is an important parameter that states the size of the REDO log files in the node. REDO log files are organised in a ring such that it is important that the tail and the head doesn't meet. When the tail and head have come to close the each other the node will start aborting all updating transactions since there is no room for the log records. REDO log records aren't removed until three local checkpoints have completed since the log record was inserted. The speed of checkpoint is controlled by a set of other parameters so these parameters are all glued together. The default parameter is 8 which means 8 sets of 4 16 MByte files. Thus in total 512 MBytes. Thus the unit is 64 MByte of REDO log space. In high update scenarios this parameter needs to be set very high. Test cases where it has been necessary to set it to over 300 have been performed.
[DB]MaxNoOfSavedMessages
This parameter sets the maximum number of trace files that will be kept before overwriting old trace files. Trace files are generated when the node crashes for some reason. Default is 25 trace files.

The next set of parameters defines the pool sizes for meta data objects. It is necessary to define the maximum number of attributes, tables, indexes and trigger objects used by indexes, events and replication between clusters.

[DB]MaxNoOfAttributes
This parameter defines the number of attributes that can be defined in the cluster. Default value of this parameter is 1000. The minimum value is 32 and the maximum is 4096.
[DB]MaxNoOfTables
A table object is allocated for each table, for each unique hash index and for each ordered index. This parameter sets the maximum number of table objects in the cluster. Default value of this parameter is 32. The minimum is 32 and the maximum is 128.
[DB]MaxNoOfIndexes
This parameter is only used by unique hash indexes. There needs to be one record in this pool for each unique hash index defined in the cluster. Default value of this parameter is 128.
[DB]MaxNoOfTriggers
For each unique hash index an internal update, insert and delete trigger is allocated. Thus three triggers per unique hash index. Ordered indexes use only one trigger object. Backups also use three trigger objects for each normal table in the cluster. When replication between clusters is supported it will also use internal triggers. This parameter sets the maximum number of trigger objects in the cluster. Default value of this parameter is 768. The maximum value is 2432.

There is a set of boolean parameters affecting the behaviour of storage nodes. Boolean parameters can be specified to true by setting it to Y or 1 and to false by setting it to N or 0.

[DB]LockPagesInMainMemory
For a number of operating systems such as Solaris and Linux it is possible to lock a process into memory and avoid all swapping problems. This is an important feature to provide real-time characteristics of the cluster. Default is that this feature is not enabled.
[DB]StopOnError
This parameter states whether the process is to exit on error condition or whether it is perform an automatic restart. Default is that this feature is enabled.
[DB]RestartOnErrorInsert
This feature is only accessible when building the debug version where it is possible to insert errors in the execution of various code parts to test failure cases. Default is that this feature is not enabled.

There are quite a few parameters specifying time-outs and time intervals between various actions in the storage nodes. Most of the time-outs are specified in milliseconds with a few exceptions which will be mentioned below.

[DB]TimeBetweenWatchDogCheck
To ensure that the main thread doesn't get stuck in an eternal loop somewhere there is a watch dog thread which checks the main thread. This parameter states the number of milliseconds between each check. After three checks and still being in the same state the process is stopped by the watch dog thread. This parameter can easily be changed and can be different in the nodes although there seems to be little reason for such a difference. The default time-out is 4000 milliseconds or 4 seconds.
[DB]StartPartialTimeout
This parameter specifies the time that the cluster will wait for all storage nodes to come up before the algorithm to start the cluster is invocated. This time out is used to avoid starting only a partial cluster if possible. The default value is 30000 milliseconds, 30 seconds. 0 means eternal time out. Thus only start if all nodes are available.
[DB]StartPartitionedTimeout
If the cluster is ready start after waiting StartPartialTimeout but is still in a possibly partitioned state one waits until also this timeout has passed. The default timeout is 60000 milliseconds, 60 seconds.
[DB]StartFailureTimeout
If the start is not completed within the time specified by this parameter the node start will fail. Setting this parameter to 0 means no time out is applied on the time to start the cluster. The default value is 60000 milliseconds, 60 seconds. For storage nodes containing large data sets this parameter needs to be increased since it could very well take 10-15 minutes to perform a node restart of a storage node with a few GBytes of data.
[DB]HeartbeatIntervalDbDb
One of the main methods of discovering failed nodes is by heartbeats. This parameter states how often heartbeat signals are sent and how often to expect to receive them. After missing three heartbeat intervals in a row, the node is declared dead. Thus the maximum time of discovering a failure through the heartbeat mechanism is four times the heartbeat interval. The default heartbeat interval is 1500 milliseconds, 1.5 seconds. This parameter must not be changed drastically. If one node uses 5000 milliseconds and the node watching it uses 1000 milliseconds then obviously the node will be declared dead very quickly. So this parameter can be changed in small steps but not in large steps.
[DB]HeartbeatIntervalDbApi
In a similar manner each storage node sends heartbeats to each of the connected MySQL servers to ensure that they behave properly. If a MySQL server doesn't send a heartbeat in time (same algorithm as for storage node with three heartbeats missed causing failure) it is declared down and all ongoing transactions will be completed and all resources will be released and the MySQL server cannot reconnect until the completion of all activities started by the previous MySQL instance has been completed. The deafult interval is 1500 milliseconds. This interval can be different in the storage node since each storage node independently of all other storage nodes watches the MySQL servers connected to it.
[DB]TimeBetweenLocalCheckpoints
This parameter is an exception in that it doesn't state any time to wait before starting a new local checkpoint. This parameter is used to ensure that in a cluster where not so many updates are taking place that we don't perform local checkpoints. In most clusters with high update rates it is likely that a new local checkpoint is started immediately after the previous was completed. The size of all write operations executed since the start of the previous local checkpoints is added. This parameter is specified as the logarithm of the number of words. So the default value 20 means 4 MByte of write operations, 21 would mean 8 MByte and so forth up until the maximum value 31 which means 8 GByte of write operations. All the write operations in the cluster are added together. Setting it to 6 or lower means that local checkpoints will execute continuosly without any wait between them independent of the workload in the cluster.
[DB]TimeBetweenGlobalCheckpoints
When a transaction is committed it is committed in main memory in all nodes where mirrors of the data existed. The log records of the transaction are not forced to disk as part of the commit however. The reasoning here is that having the transaction safely committed in at least two independent computers should be meeting standards of durability. At the same time it is also important to ensure that even the worst of cases when the cluster completely crashes is handled properly. To ensure this all transactions in a certain interval is put into a global checkpoint. A global checkpoint is very similar to a grouped commit of transactions. An entire group of transactions is sent to disk. Thus as part of the commit the transaction was put into a global checkpoint group. Later this groups log records are forced to disk and then the entire group of transaction is safely committed also on all computers disk storage as well. This parameter states the interval between global checkpoints. The default time is 2000 milliseconds.
[DB]TimeBetweenInactiveTransactionAbortCheck
Time-out handling is performed by checking each timer on each transaction every period of time in accordance with this parameter. Thus if this parameter is set to 1000 milliseconds, then every transaction will be checked for time-out once every second. The default for this parameter is 1000 milliseconds.
[DB]TransactionInactiveTimeout
If the transaction is currently not performing any queries but is waiting for further user input, this parameter states the maximum time that the user can wait before the transaction is aborted. The default for this parameter is 3000 milliseconds. Obviously for a database used interactively without autocommit this parameter needs to changed from its default value.
[DB]TransactionDeadlockDetectionTimeout
When a transactioon is involved in executing a query it waits for other nodes. If the other nodes doesn't respond it could depend on three things. First, the node could be dead, second the operation could have entered a lock queue and finally the node requested to perform the action could be heavily overloaded. This time-out parameter states how long the transaction coordinator will wait until it aborts the transaction when waiting for query execution of another node. Thus this parameter is important both for node failure handling and for deadlock detection. Setting it too high would cause a non-desirable behaviour at deadlocks and node failures. The default time out is 3000 milliseconds, 3 seconds.
[DB]NoOfDiskPagesToDiskAfterRestartTUP
When executing a local checkpoint the algorithm sends all data pages to disk during the local checkpoint. Simply sending them there as quickly as possible will cause unnecessary load on both processors, networks, and disks. Thus to control the write speed this parameter specifies how many pages per 100 milliseconds is to be written. A page is here defined as 8 kBytes. The unit this parameter is specified in is thus 80 kBytes per second. So setting it to 20 means writing 1.6 MBytes of data pages to disk per second during a local checkpoint. Also writing of UNDO log records for data pages is part of this sum. Writing of index pages (see IndexMemory to understand what index pages are used for) and their UNDO log records is handled by the parameter NoOfDiskPagesToDiskAfterRestartACC. This parameter handles the limitation of writes from the DataMemory. So this parameter specifies how quickly local checkpoints will be executed. This parameter is important in connection with NoOfFragmentLogFiles, DataMemory, IndexMemory. The default value is 10, thus 800 kBytes of data pages per second.
[DB]NoOfDiskPagesToDiskAfterRestartACC
This parameter has the same unit as NoOfDiskPagesToDiskAfterRestartTUP but limits the speed of writing index pages from IndexMemory. The default value of this parameter is 5, thus 400 kBytes per second.
[DB]NoOfDiskPagesToDiskDuringRestartTUP
This parameter specifies the same things as NoOfDiskPagesToDiskAfterRestartTUP and NoOfDiskPagesToDiskAfterRestartACC, only it does it for local checkpoints executed in the node as part of a local checkpoint when the node is restarting. As part of all node restarts a local checkpoint is always performed. Since during a node restart it is possible to use a higher speed of writing to disk since less activities is performed in the node due to the restart phase. This parameter handles the DataMemory part. The default value is 40, thus 6.4 MBytes per second.
[DB]NoOfDiskPagesToDiskDuringRestartACC
During Restart for IndexMemory part of local checkpoint. The default value is 20, thus 3.2 MBytes per second.
[DB]ArbitrationTimeout
This parameter specifies the time that the storage node will wait for a response from the arbitrator when sending an arbitration message in the case of a split network. The default value is 1000 milliseconds, 1 second.

For management of the cluster it is important to be able to control the amount of log messages sent to stdout for various event types. The possible events will be listed in this manual soon. There are 16 levels possible from level 0 to level 15. Setting event reporting to level 15 means receiving all event reports of that category and setting it to 0 means getting no event reports in that category.

The reason why most defaults are set to 0 and thus not causing any output to stdout is that the same message is sent to the cluster log in the management server. Only the start-up message is by default generated to stdout.

A similar set of levels can be set in management client to define what levels to record in the cluster log.

[DB]LogLevelStartup
Events generated during startup of the process. The default level is 1.
[DB]LogLevelShutdown
Events generated as part of graceful shutdown of a node. The default level is 0.
[DB]LogLevelStatistic
Statitical events such as how many primary key reads, updates, inserts and many other statistical information of buffer usage and so forth. The default level is 0.
[DB]LogLevelCheckpoint
Events generated by local and global checkpoints. The default level is 0.
[DB]LogLevelNodeRestart
Events generated during node restart. The default level is 0.
[DB]LogLevelConnection
Events generated by connection between nodes in the cluster. The default level is 0.
[DB]LogLevelError
Events generated by errors and warnings in the cluster. These are errors not causing a node failure but still considered an error worthy of reporting it. The default level is 0.
[DB]LogLevelInfo
Events generated for information about state of cluster and so forth. The default level is 0.

There is a set of parameters defining memory buffers set aside for on-line backup execution.

[DB]BackupDataBufferSize
When executing a backup there are two buffers used for sending data to the disk. This buffer is used to fill in data recorded by scanning the tables in the node. When filling this to a certain level the pages are sent to disk. This level is specified by the parameter BackupWriteSize. When sending data to the disk, the backup can continue filling this buffer until it runs out of buffer space. When running out of buffer space it will simply stop the scan and wait until some disk writes return and thus frees up memory buffers to use for further scanning. The default is 2 MBytes.
[DB]BackupLogBufferSize
This parameter has a similar role but instead used for writing a log of all writes to the tables during execution of the backup. The same principles apply for writing those pages as for BackupDataBufferSize except that when this part runs out of buffer space it will cause the backup to fail due to lack of backup buffers. Thus the size of this buffer must be big enough to handle the load caused by write activities during the backup execution. The default parameter should be big enough. Actually it is more likely that a backup failure is caused by a disk not able to write as quickly as it should. If the disk subsystem is not dimensioned for the write load caused by the applications this will create a cluster which will have great difficulties to perform the desired actions. It is important to dimension the nodes in such a manner that the processors becomes the bottleneck rather than the disks or the network connections. The default is 2 MBytes.
[DB]BackupMemory
This parameter is simply the sum of the two previous, the BackupDataBufferSize and BackupLogBufferSize. The default is 4 MBytes.
[DB]BackupWriteSize
This parameter specifies the size of the write messages to disk for the log and data buffer used for backups. The default is 32 kBytes.

17.3.4.5 Defining the MySQL Servers in a MySQL Cluster

The API section defines the behaviour of the MySQL server. Id, ArbitrationRank and ExecuteOnComputer are mandatory.

[API]Id
This identity is the node id used as the address of the node in all cluster internal messages. This is an integer between 1 and 63. Each node in the cluster must have a unique identity.
[API]ExecuteOnComputer
This is referring to one of the computers defined in the computer section.
[API]ArbitrationRank
This parameter is used to define which nodes can act as an arbitrator. MGM nodes and API nodes can be arbitrators. 0 means it isn't used as arbitrator, 1 high priority and 2 low priority. A normal configuration uses the management server as arbitrator setting the ArbitrationRank to 1 (which is the default) and setting all APIs to 0 (not default in MySQL 4.1.3).
[API]ArbitrationDelay
If setting this to anything else than 0 it means that the management server will delay responses to the arbitration requests. Default is no delay and this should not be necessary to change.

17.3.4.6 Defining TCP/IP connections in a MySQL Cluster

TCP/IP connections is the default transport mechanism for MySQL Cluster. It is actually not necessary to define any connection since there will be a one connection set-up between each of the storage node, between each storage node and all MySQL server nodes and between each storage node and the management server.

It is only necessary to define a connection if it is necessary to change the default values of the connection. In that case it is necessary to define at least NodeId1, NodeId2 and the parameters to change.

It is also possible to change the default values by setting the parameters in the TCP DEFAULT section.

[TCP]NodeId1
[TCP]NodeId2
To identify a connection between two nodes it is necessary to provide the node identity for both of them in NodeId1 and NodeId2.
[TCP]SendBufferMemory
TCP transporters use a buffer all messages before performing the send call to the operating system. When this buffer reaches 64 kByte it sends the buffer, the buffer is also sent when a round of messages have been executed. To handle temporary overload situations it is also possible to defined a bigger send buffer. The default size of the send buffer is 256 kByte.
[TCP]SendSignalId
To be able to retrace a distributed message diagram it is necessary to identify each message with an identity. By setting this parameter to Y these message identities are also transported over the network. This feature is not enabled by default.
[TCP]Checksum
This parameter is also a Y/N parameter which is not enabled by default. When enabled all messages are checksummed before put into the send buffer. This feature enables control that messages are not corrupted while waiting in the send buffer. It is also a double check that the transport mechanism haven't corrupted the data.
[TCP]PortNumber
This is the port number to use for listening to connections from other nodes. This port number should be specified in the TCP DEFAULT section normally.
[TCP]ReceiveBufferMemory
This parameter specifies the size of the buffer used when receiving data from the TCP/IP socket. There is seldom any need of changing this from its default value of 64 kBytes. One possible reason could be to save memory.

17.3.4.7 Defining shared-memory connections in a MySQL Cluster

Shared memory segments are currently only supported for special builds of MySQL Cluster using the configure parameter --with-ndb-shm. Its implementation will most likely change. When defining shared memory as the connection method it is necessary to define at least NodeId1, NodeId2 and ShmKey. All other parameters have default values which will work out fine in most cases.

[SHM]NodeId1
[SHM]NodeId2
To identify a connection between two nodes it is necessary to provide the node identity for both of them in NodeId1 and NodeId2.
[SHM]ShmKey
When setting up shared memory segments an identifier is used to uniquely identify the shared memory segment to use for the communication. This is an integer which does not have a default value.
[SHM]ShmSize
Each connection has a shared memory segment where messages between the nodes are put by the sender and read by the reader. This segment has a size defined by this parameter. Default value is 1 MByte.
[SHM]SendSignalId
To be able to retrace a distributed message diagram it is necessary to identify each message with an identity. By setting this parameter to Y these message identities are also transported over the network. This feature is not enabled by default.
[SHM]Checksum
This parameter is also a Y/N parameter which is not enabled by default. When enabled all messages are checksummed before put into the send buffer. This feature enables control that messages are not corrupted while waiting in the send buffer. It is also a double check that the transport mechanism haven't corrupted the data.

17.4 Process Management in MySQL Cluster

There are four processes that are important to know about when using MySQL Cluster. We will cover how to work with those processes, which options to use when starting and so forth.

17.4.1 MySQL Server Process Usage for MySQL Cluster

@command{mysqld} is the traditional MySQL server process. To be used with MySQL Cluster it needs to be built with support for the NDBCluster storage engine. If the @command{mysqld} binary has been built in such a manner, the NDBCluster storage engine is still disabled by default.

To enable the NDBCluster storage engine there are two ways. Either use --ndbcluster as a start-up option when starting @command{mysqld} or insert a line with ndbcluster in the [mysqld] section of your `my.cnf' file. An easy way to verify that your server runs with support for the NDBCluster storage engine is to issue the command SHOW TABLE TYPES from a mysql client. You should see YES for the row listing NDBCLUSTER. If you see NO, you are not running a @command{mysqld} that is compiled with NDBCluster support enabled. If you see DISABLED, then you need to enable it in the my.cnf configuration file.

The MySQL server needs to know how to get the configuration of the cluster. To access this configuration it needs to know three things, it needs to know its own node id in the cluster, it needs to know the hostname (or IP address) where the management server resides and finally it needs to know the port on which it can connect to the management server.

There are currently two possible ways to provide this information to the @command{mysqld} process. The first option is to include this information in a file called `Ndb.cfg'. This file should reside in the data directory of the MySQL Server. The second option is to set an environment variable called NDB_CONNECTSTRING. The string is the same in both cases.

host=ndb_mgmd.mysql.com:2200

ndb_mgmd.mysql.com is the host where the management server resides, and it is listening to port 2200.

With this set-up the MySQL server will be a full citizen of MySQL Cluster and will be fully aware of all storage nodes in the cluster and their status. It will set-up connection to all storage engine nodes and will be able to use all storage engine nodes as transaction coordinator and to access their data for reading and updating.

17.4.2 ndbd, the Storage Engine Node Process

ndbd is the process which is used to handle all the data in the tables using the NDBCluster storage engine. This is the process that contains all the logic of distributed transaction handling, node recovery, checkpointing to disk, on-line backup and lots of other functionality.

In a cluster there is a set of ndbd processes cooperating in handling the data. These processes can execute on the same computer or on different computers, in a completely configurable manner.

Each ndbd process should start from a separate directory. The reason for this is that ndbd generates a set of log files in its starting directory. These log files are:

It is recommended to not use a directory mounted through NFS since that can in some environments cause problems with the lock on the pid-file remaining even after the process has stopped.

Also when starting the ndbd process it is necessary to specify which node id the process is to use, the host of the management server and the port it is listening to. Again there are two ways of specifying this information. Either in a string in the file `Ndb.cfg', this file should be stored in the starting directory of the ndbd process. The second option is to set the environment variable NDB_CONNECTSTRING before starting the process.

When ndbd starts it will actually start two processes. The starting process is called the "angel" and its only job is to discover when the execution process has completed and then if configured to do so, to restart the ndbd process. Thus if one attempts to kill the ndbd through the kill command in Unix, it is necessary to kill both processes.

The execution process will use one thread for all activities in reading, writing and scanning data and all other activities. This thread is designed with asynchronous programming so it can easily handle thousands of concurrent activites. In addition there is a watch-dog thread supervising the execution thread to ensure it doesn't stop in an eternal loop or other problem. There is a pool of threads handling file I/O. Each thread can handle one open file. In addition threads can be used for connection activities of the transporters in the ndbd process. Thus in a system that performs a large number of activities including update activities the ndbd process will consume up to about 2 CPUs if allowed to. Thus in a large SMP box with many CPUs it is recommended to use several ndbd processes which are configured to be part of different node groups.

nodeid=2;host=ndb_mgmd.mysql.com:2200

17.4.3 ndb_mgmd, the Management Server Process

The management server is the process which reads the configuration file of the cluster and distributes this information to all nodes in the cluster requesting it. It does also maintain the log of cluster activities. Management clients can connect to the management server and use commands to check status of the cluster in various aspects.

Also when starting ndb_mgmd it is necessary to state the same information as for ndbd and @command{mysqld} processes and again there are two options using the file `Ndb.cfg' or using the environment variable NDB_CONNECTSTRING. The `Ndb.cfg' will if used be placed in the start directory of ndb_mgmd.

nodeid=1;host=ndb_mgmd.mysql.com:2200

The following files are created or used by ndb_mgmd in its starting directory of ndb_mgmd:

17.4.4 ndb_mgm, the Management Client Process

The final important process to know about is the management client. This process is not needed to run the cluster. Its value lies in its ability to check status of the cluster, start backups and other management activities. It does so by providing access to a set of commands.

Actually the management client is using a C API which is used to access the management server so for advanced users it is also possible to program dedicated management processes which can do similar things as the management client can do.

When starting the management client it is necessary to state the hostname and port of the management server as in the example below.

ndb_mgm localhost 2200

17.4.5 Command Options for MySQL Cluster Processes

17.4.5.1 MySQL Cluster-Related Command Options for @command{mysqld}

--ndbcluster
If the binary includes support for the NDBCluster storage engine the default disabling of support for the NDB storage engine can be overruled by using this option. Using the NDBCluster storage engine is necessary for using MySQL Cluster.
--skip-ndbcluster
Disable the NDBCluster storage engine. This is disabled by default for binaries where it is included. So this option only applies if the server was configured to use the NDBCluster storage engine.

17.4.5.2 Command Options for ndbd

-?, --usage
These options only starts the program to print its command options.
-c string, --connect-string string
For ndbd it is also possible to set the connect string to the management server as a command option.
nodeid=2;host=ndb_mgmd.mysql.com:2200
-d, --daemon
Instructs ndbd to execute as a daemon process.
-i, --initial
Instructs ndbd to perform an initial start. An initial start will erase any files created by earlier ndbd instances for recovery. It will also recreate recovery log files which on some Operating Systems can take a substantial amount of time.
-n, --no-start
Instructs ndbd to not automatically start. ndbd will connect to the management server and get the configuration and initialise communication objects. It will not start the execution engine until requested to do so by the management server. The management server can request by command issued by the management client.
-s, --start
Instructs the ndbd process to immediately start. This is the default behavior so it is not really needed.
-v, --version
Prints the version number of the ndbd process. The version number is the MySQL Cluster version number. This version number is important since at start-up the MySQL Cluster processes verifies that the versions of the processes in the cluster can co-exist in the cluster. It is also important for on-line software upgrade of MySQL Cluster (see section Software Upgrade of MySQL Cluster).

17.4.5.3 Command Options for ndb_mgmd

-?, --usage
These options only starts the program to print its command options.
-c filename
Instructs the management server which file to use as configuration file. This option must be specified. There is no default value.
-d
Instructs ndb_mgmd to start as a daemon process.
-l filename
Instructs the management server in which file it can find the connect string. Default file is `Ndb.cfg'.
-n
Instructs the management server to not start as a daemon process. This is the default behavior but this is quite likely to change.
--version
Prints the version number of the management server. The version number is the MySQL Cluster version number. The management server can check that only versions capable of working with its versions are accepted and provided with the configuration information.

17.4.5.4 Command Options for ndb_mgm

-?, --usage
These options only starts the program to print its command options.
[host_name [port_num]]
To start the management client it is necessary to specify where the management server resides. This means specifying the hostname and the port. The default hostname is localhost and the default port is 2200.
--try-reconnect=number
If the connection to the management server is broken it is possible to perform only a specified amount of retries before reporting a fault code to the user. Default is that it keeps retrying every 5 seconds until it succeeds.

17.5 Management of MySQL Cluster

Managing a MySQL Cluster involves a number of activities. The first activity is to configure and start-up MySQL Cluster. This is covered by the sections section 17.3 MySQL Cluster Configuration and section 17.4 Process Management in MySQL Cluster. This section covers how to manage a running MySQL Cluster.

There are essentially two ways of actively managing a running MySQL Cluster. The first is by commands entered into the management client where status of cluster can be checked, log levels changed, backups started and stopped and nodes can be stopped and started. The second method is to study the output in the cluster log. The cluster log is directed towards the cluster.log in the directory where the management server started. The cluster log contains event reports generated from the ndbd processes in the cluster. It is also possible to send the cluster log entries to a Unix system log.


Go to the first, previous, next, last section, table of contents.