Package CedarBackup2 :: Module peer :: Class RemotePeer
[show private | hide private]
[frames | no frames]

Type RemotePeer

object --+
         |
        RemotePeer


Backup peer representing a remote peer in a backup pool.

This is a class representing a remote (networked) peer in a backup pool. Remote peers are backed up using an rcp-compatible copy command. A remote peer has associated with it a name (which must be a valid hostname), a collect directory, a working directory and a copy method (an rcp-compatible command).

You can also set an optional local user value. This username will be used as the local user for any remote copies that are required. It can only be used if the root user is executing the backup. The root user will su to the local user and execute the remote copies as that user.

The copy method is associated with the peer and not with the actual request to copy, because we can envision that each remote host might have a different connect method.

The public methods other than the constructor are part of a "backup peer" interface shared with the LocalPeer class.
Method Summary
  __init__(self, name, collectDir, workingDir, remoteUser, rcpCommand, localUser)
Initializes a remote backup peer.
  stagePeer(self, targetDir, ownership, permissions)
Stages data from the peer into the indicated local target directory.
  checkCollectIndicator(self, collectIndicator)
Checks the collect indicator in the peer's staging directory.
  writeStageIndicator(self, stageIndicator)
Writes the stage indicator in the peer's staging directory.
  _getDirContents(path)
Returns the contents of a directory in terms of a Set. (Static method)
  _copyRemoteDir(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceDir, targetDir, ownership, permissions)
Copies files from the source directory to the target directory. (Static method)
  _copyRemoteFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, ownership, permissions, overwrite)
Copies a remote source file to a target file. (Static method)
  _pushLocalFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, overwrite)
Copies a local source file to a remote host. (Static method)
  _getCollectDir(self)
Property target used to get the collect directory.
  _getLocalUser(self)
Property target used to get the local user.
  _getName(self)
Property target used to get the peer name.
  _getRcpCommand(self)
Property target used to get the rcp command.
  _getRemoteUser(self)
Property target used to get the remote user.
  _getWorkingDir(self)
Property target used to get the working directory.
  _setCollectDir(self, value)
Property target used to set the collect directory.
  _setLocalUser(self, value)
Property target used to set the local user.
  _setName(self, value)
Property target used to set the peer name.
  _setRcpCommand(self, value)
Property target to set the rcp command.
  _setRemoteUser(self, value)
Property target used to set the remote user.
  _setWorkingDir(self, value)
Property target used to set the working directory.
    Inherited from object
  __delattr__(...)
x.__delattr__('name') <==> del x.name
  __getattribute__(...)
x.__getattribute__('name') <==> x.name
  __hash__(x)
x.__hash__() <==> hash(x)
  __new__(T, S, ...)
T.__new__(S, ...) -> a new object with type S, a subtype of T
  __reduce__(...)
helper for pickle
  __reduce_ex__(...)
helper for pickle
  __repr__(x)
x.__repr__() <==> repr(x)
  __setattr__(...)
x.__setattr__('name', value) <==> x.name = value
  __str__(x)
x.__str__() <==> str(x)

Property Summary
  name: Name of the peer (a valid DNS hostname).
  collectDir: Path to the peer's collect directory (an absolute local path).
  remoteUser: Name of the Cedar Backup user on the remote peer.
  rcpCommand: An rcp-compatible copy command to use for copying files.
  localUser: Name of the Cedar Backup user on the current host.
  workingDir: Path to the peer's working directory (an absolute local path).

Instance Method Details

__init__(self, name, collectDir, workingDir, remoteUser, rcpCommand=None, localUser=None)
(Constructor)

Initializes a remote backup peer.
Parameters:
name - Name of the backup peer
           (type=String, must be a valid DNS hostname)
collectDir - Path to the peer's collect directory
           (type=String representing an absolute path on the remote peer)
workingDir - Working directory that can be used to create temporary files, etc.
           (type=String representing an absolute path on the current host.)
remoteUser - Name of the Cedar Backup user on the remote peer
           (type=String representing a username, valid via the copy command)
rcpCommand - An rcp-compatible copy command to use for copying files from the peer
           (type=String representing a system command including required arguments)
localUser - Name of the Cedar Backup user on the current host
           (type=String representing a username, valid on the current host)
Raises:
ValueError - If collect directory is not an absolute path
Overrides:
__builtin__.object.__init__

Note: If provided, the rcp command will eventually be parsed into a list of strings suitable for passing to popen2.Popen4 in order to avoid security holes related to shell interpolation. This parsing will be done by the util.splitCommandLine function. See the documentation for that function for some important notes about its limitations.

stagePeer(self, targetDir, ownership=None, permissions=None)

Stages data from the peer into the indicated local target directory.

The target directory must already exist before this method is called. If passed in, ownership and permissions will be applied to the files that are copied.
Parameters:
targetDir - Target directory to write data into
           (type=String representing a directory on disk)
ownership - Owner and group that the staged files should have
           (type=Tuple of numeric ids (uid, gid))
permissions - Permissions that the staged files should have
           (type=UNIX permissions mode, specified in octal (i.e. 0640).)
Returns:
Number of files copied from the source directory to the target directory.
Raises:
ValueError - If target directory is not a directory, does not exist or is not absolute.
ValueError - If a path cannot be encoded properly.
IOError - If there were no files to stage (i.e. the directory was empty)
IOError - If there is an IO error copying a file.
OSError - If there is an OS error copying or changing permissions on a file

Notes:

  • The returned count of copied files might be inaccurate if some of the copied files already existed in the staging directory prior to the copy taking place. We don't clear the staging directory first, because some extension might also be using it.
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • Unlike the local peer version of this method, an I/O error might or might not be raised if the directory is empty. Since we're using a remote copy method, we just don't have the fine-grained control over our exceptions that's available when we can look directly at the filesystem, and we can't control whether the remote copy method thinks an empty directory is an error.

checkCollectIndicator(self, collectIndicator=None)

Checks the collect indicator in the peer's staging directory.

When a peer has completed collecting its backup files, it will write an empty indicator file into its collect directory. This method checks to see whether that indicator has been written. If the remote copy command fails, we return False as if the file weren't there.

If you need to, you can override the name of the collect indicator file by passing in a different name.
Parameters:
collectIndicator - Name of the collect indicator file to check
           (type=String representing name of a file in the collect directory)
Returns:
Boolean true/false depending on whether the indicator exists.
Raises:
ValueError - If a path cannot be encoded properly.

Note: Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. Because of this, the implementation of this method is rather convoluted.

writeStageIndicator(self, stageIndicator=None)

Writes the stage indicator in the peer's staging directory.

When the master has completed collecting its backup files, it will write an empty indicator file into the peer's collect directory. The presence of this file implies that the staging process is complete.

If you need to, you can override the name of the stage indicator file by passing in a different name.
Parameters:
stageIndicator - Name of the indicator file to write
           (type=String representing name of a file in the collect directory)
Raises:
ValueError - If a path cannot be encoded properly.
IOError - If there is an IO error creating the file.
OSError - If there is an OS error creating or changing permissions on the file

Notes:

  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • This method's behavior is UNIX-specific. It depends on the ability of tempfile.NamedTemporaryFile to create files that can be opened more than once.

_getCollectDir(self)

Property target used to get the collect directory.

_getLocalUser(self)

Property target used to get the local user.

_getName(self)

Property target used to get the peer name.

_getRcpCommand(self)

Property target used to get the rcp command.

_getRemoteUser(self)

Property target used to get the remote user.

_getWorkingDir(self)

Property target used to get the working directory.

_setCollectDir(self, value)

Property target used to set the collect directory. The value must be an absolute path and cannot be None. It does not have to exist on disk at the time of assignment.
Raises:
ValueError - If the value is None or is not an absolute path.
ValueError - If the value cannot be encoded properly.

_setLocalUser(self, value)

Property target used to set the local user. The value must be a non-empty string if it is not None.
Raises:
ValueError - If the value is an empty string.

_setName(self, value)

Property target used to set the peer name. The value must be a non-empty string and cannot be None.
Raises:
ValueError - If the value is an empty string or None.

_setRcpCommand(self, value)

Property target to set the rcp command.

The value must be a non-empty string or None. Its value is stored in the two forms: "raw" as provided by the client, and "parsed" into a list suitable for being passed to util.executeCommand via util.splitCommandLine.

However, all the caller will ever see via the property is the actual value they set (which includes seeing None, even if we translate that internally to DEF_RCP_COMMAND). Internally, we should always use self._rcpCommandList if we want the actual command list.
Raises:
ValueError - If the value is an empty string.

_setRemoteUser(self, value)

Property target used to set the remote user. The value must be a non-empty string and cannot be None.
Raises:
ValueError - If the value is an empty string or None.

_setWorkingDir(self, value)

Property target used to set the working directory. The value must be an absolute path and cannot be None.
Raises:
ValueError - If the value is None or is not an absolute path.
ValueError - If the value cannot be encoded properly.

Static Method Details

_getDirContents(path)

Returns the contents of a directory in terms of a Set.

The directory's contents are read as a FilesystemList containing only files, and then the list is converted into a sets.Set object for later use.
Parameters:
path - Directory path to get contents for
           (type=String representing a path on disk)
Returns:
Set of files in the directory
Raises:
ValueError - If path is not a directory or does not exist.

_copyRemoteDir(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceDir, targetDir, ownership=None, permissions=None)

Copies files from the source directory to the target directory.

This function is not recursive. Only the files in the directory will be copied. Ownership and permissions will be left at their default values if new values are not specified. Behavior when copying soft links from the collect directory is dependent on the behavior of the specified rcp command.
Parameters:
remoteUser - Name of the Cedar Backup user on the remote peer
           (type=String representing a username, valid via the copy command)
localUser - Name of the Cedar Backup user on the current host
           (type=String representing a username, valid on the current host)
remoteHost - Hostname of the remote peer
           (type=String representing a hostname, accessible via the copy command)
rcpCommand - An rcp-compatible copy command to use for copying files from the peer
           (type=String representing a system command including required arguments)
rcpCommandList - An rcp-compatible copy command to use for copying files
           (type=Command as a list to be passed to util.executeCommand)
sourceDir - Source directory
           (type=String representing a directory on disk)
targetDir - Target directory
           (type=String representing a directory on disk)
ownership - Owner and group that the copied files should have
           (type=Tuple of numeric ids (uid, gid))
permissions - Permissions that the staged files should have
           (type=UNIX permissions mode, specified in octal (i.e. 0640).)
Returns:
Number of files copied from the source directory to the target directory.
Raises:
ValueError - If source or target is not a directory or does not exist.
IOError - If there is an IO error copying the files.

Notes:

  • The returned count of copied files might be inaccurate if some of the copied files already existed in the staging directory prior to the copy taking place. We don't clear the staging directory first, because some extension might also be using it.
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • We don't have a good way of knowing exactly what files we copied down from the remote peer, unless we want to parse the output of the rcp command (ugh). We could change permissions on everything in the target directory, but that's kind of ugly too. Instead, we use Python's set functionality to figure out what files were added while we executed the rcp command. This isn't perfect - for instance, it's not correct if someone else is messing with the directory at the same time we're doing the remote copy - but it's about as good as we're going to get.
  • Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. We try to work around this by issuing IOError if we don't copy any files from the remote host.

_copyRemoteFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, ownership=None, permissions=None, overwrite=True)

Copies a remote source file to a target file.
Parameters:
remoteUser - Name of the Cedar Backup user on the remote peer
           (type=String representing a username, valid via the copy command)
localUser - Name of the Cedar Backup user on the current host
           (type=String representing a username, valid on the current host)
remoteHost - Hostname of the remote peer
           (type=String representing a hostname, accessible via the copy command)
rcpCommand - An rcp-compatible copy command to use for copying files from the peer
           (type=String representing a system command including required arguments)
rcpCommandList - An rcp-compatible copy command to use for copying files
           (type=Command as a list to be passed to util.executeCommand)
sourceFile - Source file to copy
           (type=String representing a file on disk, as an absolute path)
targetFile - Target file to create
           (type=String representing a file on disk, as an absolute path)
ownership - Owner and group that the copied should have
           (type=Tuple of numeric ids (uid, gid))
permissions - Permissions that the staged files should have
           (type=UNIX permissions mode, specified in octal (i.e. 0640).)
overwrite - Indicates whether it's OK to overwrite the target file.
           (type=Boolean true/false.)
Raises:
IOError - If the target file already exists.
IOError - If there is an IO error copying the file
OSError - If there is an OS error changing permissions on the file

Notes:

  • Internally, we have to go through and escape any spaces in the source path with double-backslash, otherwise things get screwed up. It doesn't seem to be required in the target path. I hope this is portable to various different rcp methods, but I guess it might not be (all I have to test with is OpenSSH).
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • We will not overwrite a target file that exists when this method is invoked. If the target already exists, we'll raise an exception.
  • Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. We try to work around this by issuing IOError the target file does not exist when we're done.

_pushLocalFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, overwrite=True)

Copies a local source file to a remote host.
Parameters:
remoteUser - Name of the Cedar Backup user on the remote peer
           (type=String representing a username, valid via the copy command)
localUser - Name of the Cedar Backup user on the current host
           (type=String representing a username, valid on the current host)
remoteHost - Hostname of the remote peer
           (type=String representing a hostname, accessible via the copy command)
rcpCommand - An rcp-compatible copy command to use for copying files from the peer
           (type=String representing a system command including required arguments)
rcpCommandList - An rcp-compatible copy command to use for copying files
           (type=Command as a list to be passed to util.executeCommand)
sourceFile - Source file to copy
           (type=String representing a file on disk, as an absolute path)
targetFile - Target file to create
           (type=String representing a file on disk, as an absolute path)
overwrite - Indicates whether it's OK to overwrite the target file.
           (type=Boolean true/false.)
Raises:
IOError - If there is an IO error copying the file
OSError - If there is an OS error changing permissions on the file

Notes:

  • We will not overwrite a target file that exists when this method is invoked. If the target already exists, we'll raise an exception.
  • Internally, we have to go through and escape any spaces in the source and target paths with double-backslash, otherwise things get screwed up. I hope this is portable to various different rcp methods, but I guess it might not be (all I have to test with is OpenSSH).
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.


Property Details

name

Name of the peer (a valid DNS hostname).
Get Method:
_getName(self)
Set Method:
_setName(self, value)

collectDir

Path to the peer's collect directory (an absolute local path).
Get Method:
_getCollectDir(self)
Set Method:
_setCollectDir(self, value)

remoteUser

Name of the Cedar Backup user on the remote peer.
Get Method:
_getRemoteUser(self)
Set Method:
_setRemoteUser(self, value)

rcpCommand

An rcp-compatible copy command to use for copying files.
Get Method:
_getRcpCommand(self)
Set Method:
_setRcpCommand(self, value)

localUser

Name of the Cedar Backup user on the current host.
Get Method:
_getLocalUser(self)
Set Method:
_setLocalUser(self, value)

workingDir

Path to the peer's working directory (an absolute local path).
Get Method:
_getWorkingDir(self)
Set Method:
_setWorkingDir(self, value)

Generated by Epydoc 2.1 on Mon Dec 18 22:53:29 2006 http://epydoc.sf.net