This guide will talk about how to install, configure, and initialize the Globus Replica Location Service (RLS) software. It includes instructions for the RLS client API and server bundles, both source and binary. The server bundles include the client bundles, you do not need to install a client bundle separately if installing a server bundle.

Version 2.1.0 of RLS now supports RLIs sending softstate updates to other RLI servers, older versions only supported LRCs updating RLIs. To support this the RLI server now requires a database (whether you use bloom filters for updates or not, older versions did not if you used Bloom filters). If you are upgrading to version 2.1.0 or later from a previous version, you must recreate your RLI database using the current schema (or create it for the first time if you did not have one before). No provision has been made to migrate old RLI data to the new version, it's assumed users doing an upgrade will bring up the new RLI database and then wait for LRCs to perform their next softstate update. This will of course recreate the RLI data. Please see RLS Server Database Configuration for directions on creating the RLS databases.

• Globus Packaging Toolkit
• RLS Server Installation
• RLS Server Database Software
• IODBC Installation
• PostgreSQL Installation
• PostgreSQL ODBC Driver (psqlODBC) Installation
• MySQL Installation
• MySQL ODBC Driver (MyODBC) Installation
• RLS Server Installation
• RLS Server Database Configuration
• RLS Server Configuration
• RLS Server MDS Configuration
• RLS Server Startup
• RLS Server Testing
• RLS Client Installation

Globus Packaging Toolkit

You will need GPT installed in order to proceed. GPT is available here.

RLS Server Installation

The RLS server depends on an ODBC interface to a Relation Database Server (RDBMS). RLS was originally developed using MySQL and MyODBC (now called Connector/ODBC), however due to licensing restrictions with MyODBC RLS now uses PostgreSQL. Either RDBMS may be used (or any RDBMS that supports ODBC). The iODBC package is used to interface to the ODBC layer of whatever RDBMS you choose. The ODBC driver for PostgreSQL is a separate package called psqlODBC. If you use PostgreSQL you'll need both PostgreSQL and psqlODBC, if you use MySQL you'll need both MySQL and MyODBC (Connector/ODBC). packages:

Any RDBMS that supports an ODBC interface should work, instructions for both PostgreSQL and MySQL are provided.

The location of iODBC and the odbc.ini file must be specified before installing the RLS server. Also if you're using MySQL its top level installation directory must be specified. By default these are assumed to be in $GLOBUS_LOCATION. The following environment variables can be used to override the default locations. These should be set prior to installing the RLS server. In addition if you're building from source and wish to build the client Java API (included in the server bundles) you need to set the path to the Java Development Toolkit (JDK) version 1.4 or later.

You can use the following commands to set these variables. You only need to set these variables for RLS installation, they are not used when running RLS. This document assumes you are using the csh shell or one of its variants, if you're using sh or something similar (eg bash) you should change the setenv commands to export variable=value.

• setenv GLOBUS_IODBC_PATH $GLOBUS_LOCATION
• setenv ODBCINI $GLOBUS_LOCATION/var/odbc.ini
• setenv JAVA_HOME /usr/jdk/1.4
• setenv GLOBUS_MYSQL_PATH $GLOBUS_LOCATION # if using MySQL

RLS Server Database Software

The RLS server was originally developed using MySQL and MyODBC. However due to licensing restrictions with MyODBC it is now developed using PostgreSQL and psqlODBC. You may use either set of packages. In either case you need iODBC, which provides a generalized ODBC interface to the ODBC drivers provided with Postgres or MySQL.

This document does not purport to be an installation guide for iODBC, PostgreSQL, psqlODBC, MySQL or MyODBC. Please check the relevant web sites for complete documentation. The procedures followed by the RLS developers is briefly covered here.

iODBC Installation

The following commands were used during RLS development to install iODBC version 3.0.5. $IODBCSRC is the directory where you untarred the iODBC sources, and $ODBCINIDIR is the directory where you plan to install the odbc.ini file.

• cd $IODBCSRC
• ./configure --prefix=$GLOBUS_IODBC_PATH --disable-gtktest --with-pthreads --disable-gui --with-iodbc-inidir=$ODBCINIDIR
• gmake
• gmake install

You must now create the odbc.ini file, in $ODBCINIDIR. The contents should include the path to where you intend to install the ODBC driver for your RDBMS (eg psqlodbc.so or libmyodbc3.so). The following is an example that should work with psqlODBC, it assumes you will name your LRC and RLI databases "lrc1000"and "rli1000".

There is apparently a bug in psqlODBC, it will not find a Data Source Name (DSN) in the system odbc.ini file $ODBCINIDIR/odbc.ini. It will find DSNs in the user's odbc.ini file if it exists <$HOME/.odbc.ini). One work around is to copy or symlink the system odbc.ini file to each user's home directory. psqlODBC does find system DSNs in a file called odbcinst.ini, which is looked for in the etc subdirectory where psqlODBC was installed $PSQLODBC_INSTALL/etc/odbcinst.ini. So another option besides creating user .odbc.ini files is to copy or symlink the system odbc.ini file to $PSQLODBC_INSTALL/etc/odbcinst.ini. Someone who understands this better may have a better answer.

[ODBC Data Sources]
lrc1000=lrc database
rli1000=rli database

[lrc1000]
Description=LRC database
DSN=lrc1000
Servertype=postgres
Servername=localhost
Database=lrc1000
ReadOnly=no

[rli1000]
Description=RLI database
DSN=rli1000
Servertype=postgres
Servername=localhost
Database=rli1000
ReadOnly=no

[Default]
Driver=/path/to/psqlodbc.so
Port=5432

If you're using MySQL and changed how how MySQL clients connect to the MySQL server in my.cnf (eg the port number or socket name) then you should set option 65536 in odbc.ini for each database, this tells MyODBC to read the client section of my.cnf to find the changed connection parameters.

[lrc1000]
option = 65536

[rli1000]
option = 65536

PostgreSQL Installation

The commands used to install Postgres 7.2.3 on the RLS development system were as follows. $POSTGRESSRC is the directory where the PostgreSQL source was untarred.

• cd $POSTGRESSRC
• ./configure --prefix=$GLOBUS_LOCATION
• gmake
• gmake install

The following commands can be used to initialize PostgreSQL and start the server (the "-o -F" flags to postmaster disable fsync() calls after transactions which improves performance but risks DB corruption):

• initdb -D /path/to/postgres-datadir
• postmaster -D /path/to/postgres-datadir -i -o -F

The following command will create the database user (called "dbuser") and password that RLS will use:

• createuser -P dbuser

Please be sure to do periodic vacuum and analyze commands on all your Postgres databases. The Postgres documentation recommends doing this daily from cron. Failure to do this can seriously degrade performance, to the point where routine RLS operations (eg LRC to RLI softstate updates) timeout and fail. Please see the Postgres documentation for further details.

psqlODBC Installation

The following commands were used to install psqlODBC 7.2.5. Note the configure script that comes with psqlODBC supports a --with-iodbc option, however when the RLS developers used this it resulted in RLS servers with corrupt memory that would dump core while opening the database connection. It seems to work fine (with iODBC) without this option. $PSQLODBCSRC is the directory where you untarred the psqlODBC source, and $PSQLODBC_INSTALL is where you plan to install it:

• cd $PSQLODBCSRC
• setenv CPPFLAGS -I$(IODBC_INSTALLDIR)/include
• ./configure --prefix=$PSQLODBC_INSTALL --enable-pthreads
• gmake
• gmake install

MySQL Installation

The RLS server depends on transaction support in the RDBMS. If you're using MySQL it supports transactions beginning in version 4.0.1 or later. The commands used to install MySQL 4.0.2-alpha on the RLS development system were as follows. $MYSQLSRC is the directory where you untarred the MySQL source. Note there may be a bug in the debugging code of MySQL, it should be configured with --without-debug. There is definitely a bug in the MyODBC debugging code (see below).

• setenv CFLAGS -g
• setenv CXX gcc
• setenv CXXFLAGS "-g -felide-constructors -fno-exceptions -fno-rtti"
• cd $MYSQLSRC
• ./configure --prefix=$GLOBUS_LOCATION --localstatedir=$GLOBUS_MYSQL_PATH/var/db --without-berkeley-db --with-pthread --enable-assembler --enable-thread-safe-client --without-debug
• gmake
• gmake install
• scripts/mysql_install_db

MySQL is configured via a file called my.cnf. There are sample configuration files for configuring small, medium, large and huge databases provided with the MySQL distribution in the support-files directory. Your my.cnf file should be installed in the directory specified by localstatedir when you configured MySQL. Some Linux systems come with MySQL already installed. These typically have a global my.cnf file installed in /etc/my.cnf. If you choose to install MySQL yourself you should remove or rename that file so that it doesn't conflict with your new my.cnf file, or you can start the MySQL server with the --default-file option set to the path to your my.cnffile. You should make the following changes for RLS:

• Uncomment all the innodb options. RLS depends on the innodb table type since it's the only one that supports transactions.
• Set the innodb_data_home_dir, innodb_log_group_home_dir and innodb_log_arch_dir options to the directory(s) where you want the innodb data files to reside. On the development systems these were set to the same value as localstatedir specified to configure.
• You may want to change the size of your database, by changing the sizes on the data files in the innodb_data_file_path option. A very rough rule of thumb for RLS is to set this to at least 1000 times the maximum number of Logical File Names you expect to have in your database. There may be more than one data file, the total size of all the files should be at least 1000 times the maximum number of LFNs.
• You may want to set the innodb_flush_log_at_trx_commit option to 0, this can improve update performance dramatically, at the cost of small extra risk of a corrupt database should your host crash during an update. For example on a Linux host the time to create 1,000,000 entries in an RLS database decreased from over 10 hours to 20 minutes by turning this option off.

Once you've installed and configured MySQL you must start the database server and create the database user/password that RLS will use to connect to the database. To start the database server:

• mysqld_safe [--defaults-file=path to your my.cnf file]

To create the database user and password that RLS will use you must run the MySQL command line tool mysql, and specify the following commands. These commands assume the username you will create for RLS is dbuser with password dbpassword, and the database(s) you will create for your LRC and/or RLI server are lrc1000 and rli1000. Creation of the LRC and/or RLI databases is covered below in the section RLS Server Database Configuration

mysql> use mysql;
mysql> grant all on lrc1000.* to dbuser@localhost identified by 'dbpassword';
mysql> grant all on rli1000.* to dbuser@localhost identified by 'dbpassword';

MyODBC Installation

There is a bug in MyODBC version 3.51.05 and earlier. The debug code is not thread safe, and the RLS server will get a segmentation violation and die if this code is enabled. In versions 3.51.05 and later the debug code can be disabled with the configure option --without-debug, in earlier versions it is disabled by defining DBUG_OFF, eg:

• setenv CFLAGS -DDBUG_OFF

The following commands are what the RLS developers used to install MyODBC version 3.51.03. $MYODBCSRC is the directory where you untarred the MyODBC sources. The following commands install MyODBC in $GLOBUS_LOCATION, you may choose a different directory if you wish, by changing the --prefix option to configure below. It also assumes that iODBC was installed in $GLOBUS_LOCATION, this may be changed by changing the --with-iodbc-includes and --with-iodbc-libs options. $ODBCINIDIR is the directory where you created the odbc.ini file.

• cd $MYODBCSRC
• ./configure --prefix=$GLOBUS_LOCATION --with-mysql-libs=$GLOBUS_MYSQL_PATH/lib/mysql --with-mysql-includes=$GLOBUS_MYSQL_PATH/include/mysql --with-iodbc-includes=$GLOBUS_LOCATION/include --with-iodbc-libs=$GLOBUS_LOCATION/lib --with-odbc-ini=$ODBCINIDIR
• gmake
• gmake install

RLS Server Installation

To install the RLS server source bundle use the following commands:

• gpt-build globus-rls-server-1.1-src_bundle.tar.gz gcc32dbgpthr
• gpt-postinstall

To install the binary bundle do the following:

• gpt-install globus-rls-server-1.1-i686-pc-linux-gnu-bin.tar.gz
• gpt-postinstall

RLS Server Database Configuration

RLS server configuration is specified in $GLOBUS_LOCATION/etc/globus-rls-server.conf, please see the man page globus-rls-server(8) for complete details. Some of the configuration options (eg database user/password) are mentioned below.

Once the DBMS is installed and running, you should create a database user that the RLS server will use to connect to the DBMS. The database user and password you pick must be specified in the RLS server configuration file, as well as the name of the database(s) you will create (see below).

db_user dbuser
db_pwd dbpassword
lrc_dbname lrc1000 # optional (if LRC server)
rli_dbname rli1000 # optional (if RLI server)

You will also need to create the database(s) that the RLS server will use. If the RLS server is a Local Replica Catalog (LRC) server you will need to create the LRC database.

The installation process installs the schema files for the LRC and RLI databases in $GLOBUS_LOCATION/setup/globus. There are separate files for PostgreSQL and MySQL:

globus-rls-lrc-postgres.sql
globus-rls-rli-postgres.sql
globus-rls-lrc-mysql.sql
globus-rls-rli-mysql.sql

You may need to edit these files to set the name of the database user you created for RLS, and the names of the databases configured in $GLOBUS_LOCATION/etc/globus-rls-server.conf. By default the database user is dbuser, the LRC database name is lrc1000 and the RLI database name is rli1000.

Once the schema files are as you want them, you can create the databases with the following commands:

Use these commands for PostgreSQL. If you have existing RLS databases and you wish to recreate them use the dropdb command to delete the old ones first. This will of course destroy any old data.

• createdb -O dbuser -U dbuser -W lrc1000
• createdb -O dbuser -U dbuser -W rli1000
• psql -W -U dbuser -d lrc1000 -f $GLOBUS_LOCATION/setup/globus/globus-rls-lrc-postgres.sql
• psql -W -U dbuser -d rli1000 -f $GLOBUS_LOCATION/setup/globus/globus-rls-rli-postgres.sql

Use these commands for MySQL. The schema files include commands to old versions of the database if they exist, which will destroy existing data.

• mysql -p -u dbuser < $GLOBUS_LOCATION/setup/globus/globus-rls-lrc-mysql.sql
• mysql -p -u dbuser < $GLOBUS_LOCATION/setup/globus/globus-rls-rli-mysql.sql

RLS Server Configuration

You should review the server configuration file $GLOBUS_LOCATION/etc/globus-rls-server.conf and change any options you want. The server man page globus-rls-server(8) has complete details on all the options, a minimal configuration file for both an LRC and RLI server would be:

# Configure the database connection info
db_user dbuser
db_pwd dbpassword

# If the server is an LRC server
lrc_server true
lrc_dbname lrc1000

# If the server is an RLI server
rli_server true
rli_dbname rli1000

# Configure who can make requests of the server
acl .*: all # RE matching grid-mapfile users or DNs from x509 certs

The server uses a host certificate to identify itself to clients. By default this certificate is located in the files /etc/grid-security/hostcert.pem and /etc/grid-security/hostkey.pem. Host certificates have a distinguished name of the form /CN=host/FQDN. If the host you plan to run the RLS server is on does not have a host certificate you must obtain one from your Certificate Authority. The RLS server must be run as the same user who owns the host certificate files (typically root). The location of the host certificate files may be specified in $GLOBUS_LOCATION/etc/globus-rls-server.conf:

rlscertfile path-to-cert-file # default /etc/grid-security/hostcert.pem
rlskeyfile path-to-key-file # default /etc/grid-security/hostkey.pem

It is possible to run the RLS server without authentication, by starting it with the -N option, and using URL's of the form rlsn://server to connect to it. If authentication is enabled RLI servers must include acl configuration options that match the identities of LRC servers that update it, that grant the rli_update permission to the LRCs.

RLS Server Startup

The script $GLOBUS_LOCATION/sbin/SXXrls may be used to start or stop the server, eg

$GLOBUS_LOCATION/sbin/SXXrls start
$GLOBUS_LOCATION/sbin/SXXrls stop

RLS Server MDS Configuration

The server package includes a program called globus-rls-reporter that will report information about an RLS server to MDS. To enable MDS reporting please add the contents of the file $GLOBUS_LOCATION/setup/globus/rls-ldif.conf to the MDS configuration file $GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf. You will probably need to set your VO name in rls-ldif.conf, the default is local. The VO name is referenced twice, on the lines beginning dn: and args:. You must restart your MDS (GRIS) server after modifying $GLOBUS_LOCATION/etc/grid-info-resoruce-ldif.conf, you can use the following commands to do so:

$GLOBUS_LOCATION/sbin/SXXgris stop
$GLOBUS_LOCATION/sbin/SXXgris start

RLS Server Testing

To test that your RLS server installation is correct you may start the server in debug mode with the command:

$GLOBUS_LOCATION/bin/globus-rls-server -d [-N]

The -N option is helpful if you do not have a host certificate for the server host, or a user certificate for yourself, it disables authentication. The programs globus-rls-admin and globus-rls-cli can be used to test functionality, please see their respective man pages for details on their use. The simplest test is to ping the server:

$GLOBUS_LOCATION/bin/globus-rls-admin -p rls://serverhost

If you disabled authentication (by starting the server with the -N option), then use this command:

$GLOBUS_LOCATION/bin/globus-rls-admin -p rlsn://serverhost

The RLS source bundle includes a test subdirectory (see the BUILD/globus_rls_server-version/test directory). This includes a script called runtests that invokes a program called globus-rls-test to thoroughly test an RLS server. You will need to edit the config file test.conf to set the name of the database user and password (and possibly the path to your odbc.ini file if you have a non-standard odbc.ini). runtests will create databases called lrctest and rlitest, start up an RLS server, and run globus-rls-test. You may need to grant access to the test databases to the database user using grant commands similar to what you used to create your production database(s). See MySQL Installation.

RLS Client Installation

The server bundles (source and binary) include the RLS client bundles, you do not need to install the RLS client bundle separately if you've installed an RLS server bundle.

The RLS client bundles include a Java version of the RLS API (implemented as JNI wrappers to the C API). If you're building from source and wish to build the Java API you will need to set the environment variable JAVA_HOME to the directory where the Java Development Toolkit (JDK) version 1.4 or later is installed on your system before building the RLS client. The command to build the RLS client source bundle is:

• gpt-build globus-rls-client-1.1-src_bundle.tar.gz gcc32dbgpthr
• gpt-postinstall

To install the binary bundle do the following:

• gpt-install globus-rls-client-1.1-i686-pc-linux-gnu-bin.tar.gz
• gpt-postinstall