18 Jun 2006    caspar 20060618

1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
11. 
12. 

NAME

caspar - Makefile snippets for common tasks

SYNOPSIS

In a Makefile, do either
  include caspar/mk/caspar.mk
or
  include caspar/mk/docbook.mk
or
  include caspar/mk/pod.mk
.

DESCRIPTION

Caspar offers Makefile snippets for common tasks, like installing (configuration) files, or typesetting LaTeX, DocBook XML and DocBook SGML documents.

The typesetting functionality is delivered by docbook.mk and pod.mk. This is documented in caspar-typesetting(7).

The installing-stuff functionality is delivered by caspar.mk. (That's what the rest of the manual will talk about.) It enables one to run 'make install' from within a tree which typically holds configuration files, managed using CVS (or Subversion or any other version control system, for that matter).

It is useful in cases like this: all configuration files of some host are under version control, and, after commiting a change to CVS, you want to be able to easily install the new configuration file on the host.

With caspar, all you have to do is specify the hostname in one place, and specify the name of the target directory in each CVS directory.

USAGE

Within a CVS tree, create a file include/install.mk, with contents like e.g.
 csp_UHOST = root@some.host.somewhere
 include caspar/mk/caspar.mk
. Within each other directory of the CVS tree which holds files, create a Makefile, which looks like e.g.
 csp_DIR = /some/dir/ectory/
 include ../../include/install.mk
. If you'd like to use the install-recursive target too, in directories which hold subdirectories (but not files), you'll have to create a Makefile which looks something like
 include ../../include/install.mk
. From within the CVS tree, one can call:
  make <filename>-install
  make install
  make load
  make
  make install-recursive
Calling make install (or make) now will scp all files in the current directory to the remote location. The install-recursive target descends down the tree, and calls make install in each subdirectory.

Of course, you'll have to be able to ssh directly as root to the remote host to get this working (if you'd like to use csp_UHOST = root@some.host.somewhere). If you don't like this, and would like to have a PermitRootLogin no in your /etc/ssh/sshd_config, you can use csp_sucp(1). See below.

VARIABLES

The variables one can set in a calling Makefile are:
csp_UHOST
  
user@host, reachable using $(csp_PUSH) (which is scp by default)
csp_UHOSTS
  
space separated list of user@host items, reachable using $(csp_PUSH)
csp_DIR
  
directory on host, reachable using function $(csp_PUSH)
csp_PUSH
  
make function for pushing files to remote location. Usually, this is a wrapper around a script or program. The function will be passed 4 arguments: local filename, [user@]host, remote directory and (a possibly empty) extra argument. [user@]host will be set to all elements of $(csp_UHOSTS); directory will be set to $(csp_DIR). The fourth argument will be set to $(csp_XARG). Currently, $(csp_scp_FUNC), $(csp_cp_FUNC) and $(csp_sucp_FUNC) are supported as push plugins. If csp_PUSH is unset, the default $(csp_scp_FUNC) is used.
csp_LOAD
  
targets which should depend on the `load' target.
csp_BUILD
  
targets which should depend on the `build' target.
csp_CP
  
cp binary, just "cp" by default
csp_SCP
  
scp binary, just "scp" by default
csp_SUCP
  
script wrapping sudo in ssh, "csp_sucp" by default
csp_EXTRAFILES
  
extra files which should be installed. Can be used to include files starting with a dot.
csp_TABOOFILES
  
files which should never be installed. Set to Makefile .%.swp %~ #%# pod2htmd.tmp pod2htmi.tmp by default.
csp_TABOOFILES_ADD
  
extra files which should never be installed; added to list in csp_TABOOFILES.
csp_TABOOFILES_SKIP
  
files which should be installed, even if in initial csp_TABOOFILES list. Removed from csp_TABOOFILES list.
csp_TABOODIRS
  
directories to exclude in install-recursive target. set to CVS .svn by default.
csp_TABOODIRS_ADD, csp_TABOODIRS_SKIP
  
see csp_TABOOFILES equivalents.
The following variables might get phased out or removed soonish:
csp_CPFLAGS
  
extra arguments to pass to cp invocation, none by default
csp_SCPFLAGS
  
extra arguments to pass to scp invocation, e.g. '-i .ssh/id_rsa-root'
csp_SUH
  
ssh-reachable user@host (Secure User at Host)
csp_SUHS
  
space separated list of ssh-reachable user@host items (Secure User at HostS)
csp_CPDIR
  
directory to which we can cp(1), instead of scp (CoPy DIRectory)
csp_CPDIRS
  
space separated list of directories to which we can copy (CoPy DIRectorieS)
csp_SCPDIR
  
directory on ssh reachable host, with trailing / (Secure CoPy DIRectory)
csp_SUHDIRS
  
by default, this is the list $(csp_SUH):$(dir) for each $(dir) in $(csp_SUHS) (or where dir = $(csp_SCPDIR), in case csp_SUHS is not set). More items can get added to this list on the fly.

EXAMPLES

Some examples:

Using csp_UHOST
This is the simplest way to use caspar. Makefile is

  csp_UHOST = root@some.host.somewhere
  csp_DIR = /etc/
  include caspar/mk/caspar.mk
Now, running "make" will scp all files in the current directory to root@some.host.somewhere:/etc/.

More hosts, not scp but sudo
Makefile is

  csp_UHOSTS = root@some.host.somewhere root@some.other.host
  csp_PUSH = $(csp_sucp_FUNC)
  csp_DIR = /etc/
  include caspar/mk/caspar.mk
Now, running "make" will use csp_sucp to install all files in the current directory to both root@some.host.somewhere:/etc/ and root@some.other.host:/etc/. If a file named fstab is present in the current directory, running "make fstab-install" will just install that one file. If you need to sudo(1) to another user on the remote host, add something like
 csp_XARG = postgres
. (If such a username is not supplied, sudo (and csp_sucp) use the default: root.)

Using csp_CP and csp_LOAD
username/etc/Makefile is

  csp_UHOST = dummy
  csp_PUSH = $(csp_cp_FUNC)
  csp_DIR = $(HOME)/etc/
  csp_LOAD = crontab-load
  include ../include/install.mk

  crontab-load:
        crontab $(csp_DIR)/crontab
while ../include/install.mk is just
  include caspar/mk/caspar.mk
.

Using csp_DIR and csp_LOAD
etc/Makefile is

  csp_DIR = /etc/
  csp_LOAD = aliases-load
  include ../include/install.mk

  aliases-load:
      ssh $(csp_UHOST) "cd /etc; postalias aliases; postfix reload"
while ../include/install.mk is
  csp_UHOST = root@some.host.somewhere
  include caspar/mk/caspar.mk
If you don't like the explicit "ssh" in your Makefile, you might want to introduce a variable for that. E.g. add my_EXEC = ssh to ../include/install.mk, and write
  aliases-load:
      $(my_EXEC) $(csp_UHOST) "cd /etc; postalias aliases; postfix reload"
in etc/Makefile. This will make it easier to change to another push mechanism, once that would be needed.

Using csp_sucp_FUNC and csp_LOAD
If you'd like to use csp_sucp and want a `load' target, do something like:

 csp_PUSH = $(csp_sucp_FUNC)
 csp_UHOST = foobar.example.com
 csp_DIR = /etc/uruk/
 csp_LOAD = rc-load

 include caspar/mk/caspar.mk

 rc-load:
        ssh $(csp_UHOST) "sudo invoke-rc.d uruk force-reload"

Combining the csp_LOAD target with multiple hosts
You'll have to loop over csp_SUHS to execute load-command. Here's an example doing some preprocessing on the remote hosts too.

  csp_DIR = /etc/ssh/
  csp_UHOSTS = root@some.host.somewhere root@some.other.host
  csp_LOAD = sshd_config-load
  include caspar/mk/caspar.mk

  sshd_config-load:
      for suh in $(csp_UHOSTS); do \
        ssh $$suh "cd $(csp_DIR); \
         m4 sshd_config.m4 > sshd_config && \
         PATH=$$PATH:/sbin /etc/init.d/ssh restart"; \
      done
(You might want to use a variable like my_EXEC for ssh, see above.) (In pre caspar 20060216 syntax, one would write this as
  csp_SCPDIR = /etc/ssh/
  csp_SUHS = root@some.host.somewhere root@some.other.host
  csp_LOAD = sshd_config-load
  [...]
)

Using the csp_TABOOFILES_{ADD,SKIP} variables
Using the csp_TABOOFILES_{ADD,SKIP} variables is handy if you want to install a Makefile, instead of using it: Create Makefile just as you'd like to have it installed on the remote location. Now, create GNUmakefile as e.g.

  csp_TABOOFILES_SKIP = Makefile
  csp_TABOOFILES_ADD = GNUmakefile

  csp_DIR = /etc/foobar/
  csp_UHOST = root@some.host.somewhere
  include caspar/mk/caspar.mk

  load:
        ssh $(csp_UHOST) "make -C $(csp_DIR)"
(You might want to use a variable like my_EXEC for ssh, see above.) Now, make install and make load will do the right thing.

Using the csp_EXTRAFILES variable
Using the csp_EXTRAFILES variable is handy if you want to install files with a leading dot. E.g.:

  csp_EXTRAFILES = .bashrc
  csp_UHOST = root@some.host.somewhere
  csp_DIR =
  include caspar/mk/caspar.mk

Plugging your own install script in caspar
If your script foobar should be called as e.g.

  foobar --file=fstab --user@host=joe@some.host \
    --dir=/etc/ --debuglevel=3
then make sure your Makefile features something like
  csp_foobar_FUNC = foobar --file=$(1) --user@host=$(2) \
    --dir=$(3) --debuglevel=$(4)
  csp_PUSH = $(csp_foobar_FUNC)
  csp_XARG = 3
You can now use csp_UHOST and csp_DIR just as you're used to.

More advanced tricks
Supply e.g.

 csp_UHOSTS = root@localhost root@some.host.somewhere
in install.mk, to install on multiple hosts. Run
 make filename-install csp_UHOSTS=joe@otherhost
to install filename as joe@otherhost, instead of the default as given in install.mk. If you want to enable passing csp_UHOSTS as a shell environment variable, you'll have to use conditional assignment in your Makefile:
 csp_UHOSTS ?= root@localhost root@some.host.somewhere
This allows it to run
 % export csp_UHOSTS=foo@bar
 % make filename-install
to install on foo@bar.

When you don't want to ssh to root@some.host.somewhere directly, you could do

 sudo rsync -az /path/to/your/config_archive /etc
.

FILES

caspar/mk/caspar.mk, caspar/mk/docbook.mk, caspar/mk/pod.mk

ENVIRONMENT

For caspar.mk: csp_CP, csp_LOAD, csp_SCP, csp_UHOST, csp_PUSH, ...

BUGS

Very likely, GNU Make is not the best tool for doing the stuff caspar.mk is doing. See also TODO, distributed with the caspar package. (And online at http://mdcc.cx/pub/caspar/caspar-latest/TODO.)

TRIVIA

Caspar is named after Caspar the Friendly Ghost, since that's the title of the Daniel Johnston song I was listening to when deciding to package my homegrown scripts.

AUTHOR

Joost van Baal

SEE ALSO

caspar-typesetting(7) csp_sucp(1)

The caspar homepage is at http://mdcc.cx/caspar/ .

Lots of tools overlap (partly) with caspar in their functionality. Here's a list. A big part of it was collected by Ray Miller (http://users.ox.ac.uk/~raym/) of Oxford University, and published in the article "Configuration Management with Subversion, YAML and Perl Template Toolkit" in the SANE 2006 (http://www.sane.nl/sane2006/) conference proceedings.

The `cvs-conf' package (http://project.tuxfamily.org/cvs-conf, http://packages.debian.org/stable/utils/cvs-conf.html) probably offers about the same functionality as caspar.mk. However, this package seems largely unmaintained (between September 2002 and May 2004, at least).

docbookmk, by Michael Wiedmann (http://www.miwie.org/docbkmake/) offers probably a superset of Caspar's docbook.mk functionality.

latex-make by the LaTeX Utils project on http://gforge.inria.fr/projects/latex-utils/ seems to provide similar functionality as docbook.mk for LaTeX documents.

SUP, the Software Upgrade Protocol and it's implementation by Carnegie Mellon University offers another way to distribute (configuration)files. Beware though: between Nov 1996 and June 2004, no new release has been published. The Debian (ftp://ftp.debian.org/debian/pool/main/s/sup/) and NetBSD packages are likely still maintained, though.

cfengine (http://www.cfengine.org/), by Mark Burgess e.a., builds expert systems to administrate and configure large computer networks: it delivers a very big superset of caspar's installation mechanism.

PIKT (http://www.pikt.org/) is intended primarily for system monitoring, but does do configuration management too.

LCFG (http://www.lcfg.org/) is another configuration management system.

The Arusha Project (ARK, at http://ark.sf.net/) provides a framework for collaborative system administration.

Puppet (http://reductivelabs.com/projects/puppet), also something like a configuration management system.

Bcfg2 (http://trac.mcs.anl.gov/projects/bcfg2/) is yet another configuration management system.

quattor (http://quattor.web.cern.ch/) is a system administration toolkit for installation, configuration and management of Unix systems.

rb3 and friends, as written and used by Ray Miller e.a. at Oxford University, (http://users.ox.ac.uk/~raym/software/configuration-management/).

The svk version control system is said to be quite usable for handling configuration file management (without a separate install mechanism like caspar). See also this discussion on the Debian development list.