Table of Contents
JHBuild is a tool designed to make building collections of source packages (also known as modules). It uses “module set” files to describe the modules available to build. These files include dependency information that allows JHBuild to work out what modules need to be built and in what order to build what the user requested.
JHBuild was originally written for building GNOME, but has since been extended to make it usable with other projects. A “module set” file can be hosted on a web server, allowing people to provide build rules independent of the JHBuild.
JHBuild can build modules from a variety of sources, including:
JHBuild is not intended as a replacement for the distribution's package management system. Instead, it makes it easy to build everything into a separate install prefix so that it doesn't interfere with the rest of the system.
JHBuild takes a bit of work to set up on a system. As well as installing JHBuild's prerequisites, it is necessary to install the prerequisite tools needed to build the software in CVS (or where ever else it is stored).
Before downloading JHBuild, you should make sure you have a copy of Python >= 2.0 installed on your system. It is also essential that the Expat XML parser extension is installed. This will be the case if you are using Python >= 2.3, or had expat installed when building Python. You can check whether this is the case by running the following simple command from the Python interpreter:
>>>
import xml.parsers.expat
>>>
If this completes without an exception, then it is installed correctly.
At the moment, the only way to download JHBuild is via
CVS. This can be achieved with the following commands. They
should be run in the directory where jhbuild will be installed
(for example, ~/cvs/gnome2
).
$
cvs -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome login
Logging in to :pserver:anonymous@anoncvs.gnome.org:2401/cvs/gnome CVS password:
press enter
$
cvs -d :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome checkout jhbuild
$
This will download JHBuild into a
jhbuild
folder under the current
directory. Now to build and install it:
$
cd jhbuild
$
make
...
$
make install
...
$
If these steps complete successfully, a small shell script
should be installed in ~/bin
to start
JHBuild. If this directory is not in the PATH
,
it will need to be added (possibly by editing
~/.profile
or
~/.bashrc
).
Before JHBuild can be run, it will be necessary to set up
a ~/.jhbuildrc
file that configures how
JHBuild will behave.
The ~/.jhbuildrc
file uses Python
syntax to set a number of configuration variables for JHBuild.
A minimal configuration file might look something like
this:
moduleset = 'gnome-2.10' modules = [ 'meta-gnome-desktop' ] checkoutroot = os.path.join(os.environ['HOME'], 'cvs', 'gnome2') prefix = os.path.join(os.environ['HOME'], 'prefix') os.environ['INSTALL'] = os.path.join(os.environ['HOME'], 'bin', 'install-check')
This will get JHBuild to build the
meta-gnome-desktop module (and its
dependencies) from the gnome-2.10
module
set. It will unpack source trees to
~/cvs/gnome2
and install modules to
~/prefix
. It also sets the
INSTALL
environment variable to a program that
handles installation of headers specially in order to decrease
the work during a rebuild.
Some of configuration variables available include:
moduleset | A string or list of strings giving the name(s) of the module set(s) to use. If a module set name is a fully qualified URL, then the module set will be cached locally and regularly updated. If two module sets contain a module with the same name, then the last module set listed wins. |
modules | A list of strings giving the modules you want to build. The list of modules actually built will be recursively expanded to include all the dependencies. |
checkoutroot | The base directory where all source modules should be unpacked. |
prefix | The directory prefix to install modules to. |
os.environ | A dictionary representing the environment. This can be used to set or get environment variable values as seen in the example above. |
skip | A list of module names that should not be included when deciding what to build. One use of this variable is to use the version of a package included with the distribution instead of building it |
autogenargs | A string listing arguments that should be passed to the autogen.sh or configure scripts for modules. |
makeargs | A string listing arguments that should be passed to make. |
cvsroots | By default, JHBuild will check out code from CVS
using an anonymous CVS root. This dictionary is used to
tell JHBuild to use an alternative CVS root for a
particular repository (a developer would probably want to
do this). This variable is a dictionary where the keys
are short repository names (for example, gnome.org is used for the
GNOME CVS repository), and the values are the alternative
CVS root strings. |
svnroots | Similar to cvsroots but for
Subversion repositories. |
Before any modules can be built, it is necessary to have certain build tools installed. These include the GNU auto tools (autoconf, automake, libtool and gettext), pkg-config nad Python.
JHBuild can check if your distro has installed these tools using the sanitycheck command:
$
jhbuild sanitycheck
If this command prints any messages, these can be fixed in one of two ways:
The bootstrap command can be invoked like so:
$
jhbuild bootstrap
This will download and install all the build prerequisites. Once it is finished, the sanitycheck command should be rerun to verify that everything is in place.
The bootstrap command does not build all the packages required by these tools. If the OS does not provide those packages, then they will need to be built separately.
Some packages to check for include m4, perl and a C compiler.
Now that everything is set up, JHBuild can be used to
build some software. To build all the modules selected in the
~/.jhbuildrc
file, run the following command:
$
jhbuild build
This will download, configure, compile and install each of the modules. If an error occurs at any stage, JHBuild will present a menu asking the user what to do. These choices include dropping to a shell to fix the error, rerunning the build stage, giving up on the module (which will also cause any modules depending on it to fail), or ignore the error and continue.
It is also possible to build a different set of modules (and their dependencies) by passing their names as arguments to the build command:
$
jhbuild build gtk+
If you exit JHBuild part way through a build for some
reason, it is possible to pick up a build at a particular
package using the --start-at
option:
$
jhbuild build --start-at=pango
To build one or more modules, without their dependencies, the buildone command can be used:
$
jhbuild buildone gtk+
To get a list of the modules jhbuild will build, and the order they will be built in, use the list command:
$
jhbuild list
To get information about a particular module, the info command can be used:
$
jhbuild info gtk+
If your internet bandwidth varies, you can get JHBuild to download or update all the software it will build in one go without actually building it:
$
jhbuild update
Later on, you can tell JHBuild to build everything without downloading or updating:
$
jhbuild build --no-network
If you want to run a particular command with the same environment variables set that JHBuild uses, use the run command:
$
jhbuild run
program
To start a shell with that environment, use the shell command:
$
jhbuild shell
JHBuild uses a command line syntax similar to tools like CVS:
jhbuild
[global-options
] command [command-arguments
]
The global jhbuild options are:
-f
, --file
config
~/.jhbuildrc
.-m
, --moduleset
moduleset
Command specific options are listed below.
The bootstrap command is used to install a set of build utilities required to build most modules (eg. autoconf, automake, etc).
jhbuild bootstrap
Internally bootstrap is implemented
using the same code as build, using the
bootstrap.modules
moduleset.
The build command is used to build one or more packages, including their dependencies.
jhbuild build
[--autogen] [--clean] [--no-network] [--skip=module
...] [--start-at=module
] [-D date
] [module...]
If no module names are given on the command line, then the module list found in the configuration file will be used.
-a
, --autogen
-c
, --clean
-n
, --no-network
-s
,
--skip
=module,...
-t
,
--start-at
=module
-D
date
The buildone command is similar to build, but it does not use dependency information to expand the module list. It is useful for quickly rebuilding one or more modules.
jhbuild buildone
[--autogen] [--clean] [--no-network] [-D date
] module...
The --autogen
, --clean
,
--no-network
and -D
options
are processed the same as for build.
Unlike build, at least one module must be listed on the command line.
The dot command generates a file describing the directed graph formed by the dependencies between a set of modules. This file can then be processed using the GraphViz software to produce a nice diagram.
jhbuild dot
[module...]
If no module names are given on the command line, then the module list found in the configuration file will be used.
The output of this command can easily be piped to the dot utility to generate a postscript file:
$
jhbuild dot
modules
| dot -Tps > dependencies.ps
The info command is used to display information about one or more modules.
jhbuild info
module...
The command prints the module name, type, dependencies, dependent packages, and the time it was last installed with JHBuild. It may also print some information specific to the module type, such as the CVS repository or download URL.
The list command is used to show the expanded list of modules the build command would build.
jhbuild list
[--show-revision] [module...]
If no module names are given on the command line, then the module list found in the configuration file will be used.
-r
,
--show-revision
The run command is used to run an arbitrary command using the same environment as JHBuild uses when building modules.
jhbuild run
program
[argument
...]
If using JHBuild to build GNOME, this command can be useful in X startup scripts.
The sanitycheck command performs a number of checks to see whether the build environment is okay.
jhbuild sanitycheck
Some of the checks include:
The shell command starts the user's shell with the same environment as JHBuild uses when building modules.
jhbuild shell
This command is roughly equivalent to the following:
$
jhbuild run $SHELL
The tinderbox command is similar to build, but writes all terminal output to HTML files suitable for publishing on a website. It can be used to set up systems similar to Mozilla's Tinderbox, or Debian's Buildd.
jhbuild tinderbox
[--autogen] [--clean] [--no-network] [--output=directory
] [--skip=module
...] [--start-at=module
] [-D date
] [module...]
The --autogen
, --clean
,
--no-network
, --skip
,
--start-at
and -D
options are
processed the same as for build.
-o
,
--output
=directory
The update command is similar to build, but only performs the download or update stage for modules without building them.
jhbuild update
[--skip=module
...] [--start-at=module
] [-D date
] [module...]
The --skip
, --start-at
and -D
options are processed the same as for
build.
The updateone command is similar to update, but it does not use dependency information to expand the module list. It is useful for quickly updating one or more modules.
jhbuild updateone
[-D date
] module...
The -D
option is processed the same as
for update.
Unlike update, at least one module must be listed on the command line.
The ~/.jhbuildrc
file uses standard
Python syntax. The file is run, and the resulting variables
defined in the namespace are used to control how JHBuild acts. A
set of default values are inserted into the namespace before
running the user's configuration file.
alwaysautogen | If set to True , then always
run autogen.sh before
make even if a makefile exists. This
is equivalent to passing --always-autogen
option to JHBuild. Defaults to
False . |
autogenargs | A string containing arguments passed to the
autogen.sh script of all modules. Can
be overriden for particular modules using the
module_autogenargs
dictionary. |
builddir_pattern | A printf style formatting
pattern used to generate build directory names. This is
only used when using separate source and build trees. The
%s in the format string will be
replaced with the source directory name. Defaults to
'%s' . |
buildroot | A string giving the parent directory to place
build trees. Defaults to None , which
causes builds to be performed within the source
trees. |
checkoutroot | A string giving the directory to unpack source
trees to. Unless buildroot is set,
builds will occur in this directory too. Defaults to
~/cvs/gnome2 . |
cvsroots | A dictionary that can be used to change the CVS
roots used to check out source code. If you have a CVS
account for a particular project, you can set the
associated key to use that account rather than the
anonymous account. For example, you might want to set
'gnome.org' to
':ext: . |
branches | A dictionary that can be used to override the branch used for a particular module. This is useful if you are doing some changes on a branch of a module and want JHBuild to build that branch instead of the one listed in the module set. |
makeargs | A string listing additional arguments to be
passed to make. Defaults to
'' . |
makecheck | A boolean value specifying whether to run
make check after
make. This might be useful in a
tinderbox-style setup. Defaults to
False . |
makeclean | A boolean value specifying whether to run
make clean before
make. Defaults to
False . |
module_autogenargs | A dictionary mapping module names to strings
giving arguments to be passed to
autogen.sh. If a particular module
isn't listed in the dictionary, the global
autogenargs will be used
instead. |
module_makeargs | A dictionary mapping module names to strings
giving arguments to be passed to make.
If a particular module isn't listed in the dictionary, the
global makeargs will be used
instead. |
modules | A list of module names to build. This list will
be expanded using the dependency information found in the
module set. Defaults to [ 'meta-gnome-desktop'
] . |
moduleset | A string giving the name of the module set to
use. This can either be a short string to refer to one of
JHBuild's included module sets, or a full HTTP URL to
refer to an externally managed module set. Currently
defaults to 'gnome-2.10' , but is
usually updated as GNOME development progresses. |
nonetwork | A boolean value saying whether to access the
network or not. This affects checking out or updating CVS
modules, downloading tarballs and updating module sets.
Setting this to True is equivalent to
passing the --no-network option to
JHBuild. Defaults to
False . |
prefix | A string giving the prefix to install modules to.
Defaults to '/opt/gnome2' . |
pretty_print | A boolean value that can be used to disable
pretty printing of subprocess output. Currently there is
only support for pretty printing CVS output. You probably
only want to disable this if the pretty printing causes
problems. Defaults to True . |
skip | A list of modules to skip when expanding the list
of modules to build. This is similar to the
--skip option (in fact, the
--skip option extends this list). This
list is empty by default. |
sticky_date | If set, JHBuild will attempt to check out modules
as they existed at the given date. The date should be
given in the form
' .
Defaults to None . |
svnroots | Similar to cvsroots but for
Subversion repositories. |
tarballdir | If set, tarballs will be downloaded to this
directory instead of checkoutroot .
This is useful if you have multiple JHBuild environments
or regularly blow away your
checkoutroot and want to reduce
bandwidth usage. |
tinderbox_outputdir | A string giving the directory to store
jhbuild tinderbox output. This string
can be overridden by the --output option.
Defaults to None , so either the
command line option must be used or this variable must be
set in the configuration file. |
use_lib64 | A boolean value that specifies whether to install
libraries to lib64 directories. If
this is set,
--libdir=\${exec_prefix}/lib64 will be
passed to configure. Defaults to
True if running on
x86_64 , ppc64 or
s390x Linux, and
False on other systems. |
In addition to the above variables, there are some other things that can be set in the configuration file:
os.environ
This is dictionary represents the environment of the process (which also gets passed on to processes that JHBuild spawns).
Some environment variables you may want to set
include CFLAGS
, INSTALL
(to
use the more efficient install-check
program included with JHBuild) and
LDFLAGS
.
addpath
(envvar
,
pathname
)This will add a directory to a
PATH
-style environment variable. It will
correctly handle the case when the environment variable is
initially empty (having a stray colon at the beginning or
end of an environment variable can have unexpected
consequences).
This function has special handling for the
ACLOCAL_FLAGS
environment variable, which
expects paths to be listed in the form -I
.pathname
prependpath
(envvar
,
pathname
)After processing the configuration file, JHBuild
will alter some paths based on variables such as
prefix
(eg. adding
$prefix/bin
to the start of
PATH
).
The prependpath
function works
like addpath
, except that the
environment variable is modified after JHBuild has made
its changes to the environment.
JHBuild uses simple XML files to describe the dependencies
between modules. A RELAX-NG schema and Document Type Definition
are included with JHBuild in the modulesets/
directory. The RELAX-NG schema makes it trivial to edit module
set files using nxml-mode
in Emacs.
The toplevel element in a module set file is moduleset
element. Currently no XML
namespace is used, but in the future this might change. The
elements below the toplevel come in three types: module sources,
include statements and module definitions.
Rather than listing the full location of every module, a number of "module sources" are listed in the module set, and then referenced by name in the module definitions. As well as reducing the amount of redundant information in the module set, it makes it easy for a user to specify an alternative source for those modules (for CVS and Subversion, it is common for developers and users to use different repository access methods).
The cvsroot
element
is used to describe a CVS repository.
<cvsroot name="rootname
" [ default="yes|no
" ] root="anon-cvsroot
" password="anon-password
"/>
The name
attribute
should be a unique identifier for the CVS repository.
If default
attribute says whether this is the default module source for
this module set file.
The root
attribute
lists the CVS root used for anonymous access to this
repository, and the password
attribute gives the
password used for anonymous access.
The svnroot
element
is used to describe a Subversion repository.
<svnroot name="rootname
" [ default="yes|no
" ] href="anon-svnroot
"/>
The name
attribute
should be a unique identifier for the Subversion
repository.
If default
attribute says whether this is the default module source for
this module set file.
The href
attribute
lists the base URL for the repository. This will probably be
either a http
, https
or
svn
URL.
The arch-archive
element
is used to describe a GNU Arch archive.
<arch-archive name="archivename
" [ default="yes|no
" ] href="mirror-url
"/>
The name
attribute
should be the Arch archive name.
If default
attribute says whether this is the default module source for
this module set file.
The href
attribute
lists a public mirror URL for the archive.
JHBuild allows one module set to include the contents of
another by reference using the include
element.
<include href="uri
"/>
The href
is a URI
reference to the module set to be included, relative to the file
containing the include
element.
Only module definitions are imported from the referenced module set — module sources are not. Multiple levels of includes are allowed, but include loops are not (there isn't any code to handle loops at the moment).
There are various types of module definitions that can be used in a module set file, and the list can easily be extended. Only the most common ones will be mentioned here.
The cvsmodule
element
is used to define a module that is to be built from
CVS.
<cvsmodule module="modulename
" [ revision="branch-or-tag
" ] [ checkoutdir="directory
" ] [ root="rootname
" ] [ autogenargs="autogenargs
" ] [ makeargs="makeargs
" ] [ supports-non-srcdir-builds="yes|no
" ]> <dependencies> <dep package="modulename
"/> ... </dependencies> <suggests> <dep package="modulename
"/> ... </suggests> </cvsmodule>
The module
,
revision
and root
attributes identify the
module to check out from CVS. The checkoutdir
attribute can be used
to specify an alternative directory to check out to (by
default, the value of module
is used).
The autogenargs
,
makeargs
and supports-non-srcdir-builds
attributes are common to many different module types. The
autogenargs
attribute
lists additional arguments to be passed to
autogen.sh, and makeargs
lists additional
arguments to be passed to make. The
supports-non-srcdir-builds
attribute is used to mark modules that can't be cleanly built
using a separate source directory.
The dependencies
and
suggests
elements are used
to declare the dependencies of the module. Any modules listed
in the dependencies
element
will be added to the module list for jhbuild
build if it isn't already included, and make sure
the dependent modules are built first.
After generating the modules list, the modules listed in
the suggests
element will
be used to further sort the modules list (although it will not
pull any additional modules). This is intended for cases
where a module has an optional dependency on another
module.
The svnmodule
element
is used to define a module that is to be built from
Subversion.
<svnmodule module="modulename
" [ checkoutdir="directory
" ] [ root="rootname
" ] [ autogenargs="autogenargs
" ] [ makeargs="makeargs
" ] [ supports-non-srcdir-builds="yes|no
" ]> <dependencies> <dep package="modulename
"/> ... </dependencies> <suggests> <dep package="modulename
"/> ... </suggests> </svnmodule>
The module
attribute gives the path of the module relative to the
repository URI. All other options for this element are
processed as for cvsmodule
.
The archmodule
element
is used to define a module that is to be built from
a GNU Arch archive.
<archmodule version="modulename
" [ checkoutdir="directory
" ] [ root="rootname
" ] [ autogenargs="autogenargs
" ] [ makeargs="makeargs
" ] [ supports-non-srcdir-builds="yes|no
" ]> <dependencies> <dep package="modulename
"/> ... </dependencies> <suggests> <dep package="modulename
"/> ... </suggests> </archmodule>
The version
attribute gives the version to be checked out from the archive
specified by root
. All
other options for this element are processed as for cvsmodule
.
The tarball
element
is used to define a module that is to be built from
a tarball.
<tarball id="modulename
" [ version="version
" ] [ autogenargs="autogenargs
" ] [ makeargs="makeargs
" ] [ supports-non-srcdir-builds="yes|no
" ]> <source href="source-url
" [ size="source-size
" ] [ md5sum="source-md5sum
" ]/> <patches> <patch file="filename
" strip="level
"/> ... </patches> <dependencies> <dep package="modulename
"/> ... </dependencies> <suggests> <dep package="modulename
"/> ... </suggests> </tarball>
The id
and version
attributes are used to
identify the module.
The source
element
specifies the file to download and compile. The href
attribute is mandatory, while
the size
and md5sum
attributes are optional.
If the last two attributes are present, they are used to check
that the source package was downloaded correctly.
The patches
element
is used to specify one or more patches to apply to the source
tree after unpacking. The patch files are looked up in the
jhbuild/patches/
directory, and the
strip
attribute says how
many levels of directories to prune when applying the
patch.
The other attributes and the dependencies
and suggests
sub-elements are processed
as for cvsmodule
.
The metamodule
element defines a module that doesn't actually do anythin.
The only purpose of a module of this type is its
dependencies.
<metamodule id="modulename
"> <dependencies> <dep package="modulename
"/> ... </dependencies> <suggests> <dep package="modulename
"/> ... </suggests> </metamodule>
The id
attribute
gives the name of the module. The child elements are handled
as for cvsmodule
.
1. General JHBuild Questions | |
Q: | The wget command can't download any tarballs. How do I get it to work with my firewall? |
A: | This can be fixed by creating or editing a
ftp_proxy = http:// If you just need to use passive FTP connections (sometimes needed with NAT firewalls), add the following line: passive_ftp = on This will fix all uses of the wget command. |
Q: | Building stuff is slow. Is there any way I can speed it up? |
A: | Other than buying a faster CPU, hard disk or getting more memory, you might want to install CCache, which can cache compilation results for you. It is available with most distributions. After installing CCache, set the cache size with the following command: ccache -M 2G (replace cd ~/bin for cmd in cc gcc c++ g++; do ln -s /usr/bin/ccache $cmd done You can check the status of your cache (such as cache hit rates) with the following command: ccache -s |
Q: | Is there a better way to monitor the status of the build than looking at terminal window? |
A: | If you have Zenity >= 2.9 installed on your system, JHBuild will display an icon in the system tray. The icon will display the current build stage, and the tooltip will show the last message from JHBuild. In the future, the icon may get support for popping up a balloon on error. |
2. Building GNOME | |
Q: | What other prerequisites are needed to build GNOME with JHBuild? |
A: | Some of the packages you will need on your system include:
If you are installing distro packages, you may need to install corresponding "dev" or "devel" packages. Note that this list is just a starting point — not a comprehensive list. |
Q: | I've built GNOME with JHBuild. How do I run it? |
A: | You will want to create a
#!/bin/sh exec jhbuild run gnome-session Finally, you should make sure that the
|
Q: | I built GNOME using JHBuild with
|
A: | Don't set |
Q: | How do I get gnome-volume-manager to work when running in a prefix? |
A: | The gnome-volume-manager program reacts to messages from hald over the system message bus, which must be running as root. Assuming that your distribution comes with HAL, the main problem is getting gnome-volume-manager to talk to the system message bus. Since communication is done over a UNIX domain
socket, the easiest way to do this is to create a symlink
from mkdir -p $prefix/var/run cd $prefix/var/run ln -s /var/run/dbus dbus You may also have trouble building HAL with JHBuild,
since it tries to install some things outside of its build
root. Running |