Package CedarBackup2 :: Module peer :: Class RemotePeer
[hide private]
[frames] | no frames]

Class RemotePeer

source code

object --+
         |
        RemotePeer

Backup peer representing a remote peer in a backup pool.

This is a class representing a remote (networked) peer in a backup pool. Remote peers are backed up using an rcp-compatible copy command. A remote peer has associated with it a name (which must be a valid hostname), a collect directory, a working directory and a copy method (an rcp-compatible command).

You can also set an optional local user value. This username will be used as the local user for any remote copies that are required. It can only be used if the root user is executing the backup. The root user will su to the local user and execute the remote copies as that user.

The copy method is associated with the peer and not with the actual request to copy, because we can envision that each remote host might have a different connect method.

The public methods other than the constructor are part of a "backup peer" interface shared with the LocalPeer class.

Instance Methods [hide private]
 
__init__(self, name, collectDir, workingDir, remoteUser, rcpCommand=None, localUser=None)
Initializes a remote backup peer.
source code
 
stagePeer(self, targetDir, ownership=None, permissions=None)
Stages data from the peer into the indicated local target directory.
source code
 
checkCollectIndicator(self, collectIndicator=None)
Checks the collect indicator in the peer's staging directory.
source code
 
writeStageIndicator(self, stageIndicator=None)
Writes the stage indicator in the peer's staging directory.
source code
 
_setName(self, value)
Property target used to set the peer name.
source code
 
_getName(self)
Property target used to get the peer name.
source code
 
_setCollectDir(self, value)
Property target used to set the collect directory.
source code
 
_getCollectDir(self)
Property target used to get the collect directory.
source code
 
_setWorkingDir(self, value)
Property target used to set the working directory.
source code
 
_getWorkingDir(self)
Property target used to get the working directory.
source code
 
_setRemoteUser(self, value)
Property target used to set the remote user.
source code
 
_getRemoteUser(self)
Property target used to get the remote user.
source code
 
_setLocalUser(self, value)
Property target used to set the local user.
source code
 
_getLocalUser(self)
Property target used to get the local user.
source code
 
_setRcpCommand(self, value)
Property target to set the rcp command.
source code
 
_getRcpCommand(self)
Property target used to get the rcp command.
source code

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __str__

Static Methods [hide private]
 
_getDirContents(path)
Returns the contents of a directory in terms of a Set.
source code
 
_copyRemoteDir(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceDir, targetDir, ownership=None, permissions=None)
Copies files from the source directory to the target directory.
source code
 
_copyRemoteFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, ownership=None, permissions=None, overwrite=True)
Copies a remote source file to a target file.
source code
 
_pushLocalFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, overwrite=True)
Copies a local source file to a remote host.
source code
Properties [hide private]
  name
Name of the peer (a valid DNS hostname).
  collectDir
Path to the peer's collect directory (an absolute local path).
  remoteUser
Name of the Cedar Backup user on the remote peer.
  rcpCommand
An rcp-compatible copy command to use for copying files.
  workingDir
Path to the peer's working directory (an absolute local path).
  localUser
Name of the Cedar Backup user on the current host.

Inherited from object: __class__

Method Details [hide private]

__init__(self, name, collectDir, workingDir, remoteUser, rcpCommand=None, localUser=None)
(Constructor)

source code 
Initializes a remote backup peer.
Parameters:
  • name (String, must be a valid DNS hostname) - Name of the backup peer
  • collectDir (String representing an absolute path on the remote peer) - Path to the peer's collect directory
  • workingDir (String representing an absolute path on the current host.) - Working directory that can be used to create temporary files, etc.
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
Raises:
  • ValueError - If collect directory is not an absolute path
Overrides: object.__init__

Note: If provided, the rcp command will eventually be parsed into a list of strings suitable for passing to popen2.Popen4 in order to avoid security holes related to shell interpolation. This parsing will be done by the util.splitCommandLine function. See the documentation for that function for some important notes about its limitations.

stagePeer(self, targetDir, ownership=None, permissions=None)

source code 

Stages data from the peer into the indicated local target directory.

The target directory must already exist before this method is called. If passed in, ownership and permissions will be applied to the files that are copied.
Parameters:
  • targetDir (String representing a directory on disk) - Target directory to write data into
  • ownership (Tuple of numeric ids (uid, gid)) - Owner and group that the staged files should have
  • permissions (UNIX permissions mode, specified in octal (i.e. 0640).) - Permissions that the staged files should have
Returns:
Number of files copied from the source directory to the target directory.
Raises:
  • ValueError - If target directory is not a directory, does not exist or is not absolute.
  • ValueError - If a path cannot be encoded properly.
  • IOError - If there were no files to stage (i.e. the directory was empty)
  • IOError - If there is an IO error copying a file.
  • OSError - If there is an OS error copying or changing permissions on a file
Notes:
  • The returned count of copied files might be inaccurate if some of the copied files already existed in the staging directory prior to the copy taking place. We don't clear the staging directory first, because some extension might also be using it.
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • Unlike the local peer version of this method, an I/O error might or might not be raised if the directory is empty. Since we're using a remote copy method, we just don't have the fine-grained control over our exceptions that's available when we can look directly at the filesystem, and we can't control whether the remote copy method thinks an empty directory is an error.

checkCollectIndicator(self, collectIndicator=None)

source code 

Checks the collect indicator in the peer's staging directory.

When a peer has completed collecting its backup files, it will write an empty indicator file into its collect directory. This method checks to see whether that indicator has been written. If the remote copy command fails, we return False as if the file weren't there.

If you need to, you can override the name of the collect indicator file by passing in a different name.
Parameters:
  • collectIndicator (String representing name of a file in the collect directory) - Name of the collect indicator file to check
Returns:
Boolean true/false depending on whether the indicator exists.
Raises:
  • ValueError - If a path cannot be encoded properly.

Note: Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. Because of this, the implementation of this method is rather convoluted.

writeStageIndicator(self, stageIndicator=None)

source code 

Writes the stage indicator in the peer's staging directory.

When the master has completed collecting its backup files, it will write an empty indicator file into the peer's collect directory. The presence of this file implies that the staging process is complete.

If you need to, you can override the name of the stage indicator file by passing in a different name.
Parameters:
  • stageIndicator (String representing name of a file in the collect directory) - Name of the indicator file to write
Raises:
  • ValueError - If a path cannot be encoded properly.
  • IOError - If there is an IO error creating the file.
  • OSError - If there is an OS error creating or changing permissions on the file

Note: If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.

_getDirContents(path)
Static Method

source code 

Returns the contents of a directory in terms of a Set.

The directory's contents are read as a FilesystemList containing only files, and then the list is converted into a sets.Set object for later use.
Parameters:
  • path (String representing a path on disk) - Directory path to get contents for
Returns:
Set of files in the directory
Raises:
  • ValueError - If path is not a directory or does not exist.

_copyRemoteDir(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceDir, targetDir, ownership=None, permissions=None)
Static Method

source code 

Copies files from the source directory to the target directory.

This function is not recursive. Only the files in the directory will be copied. Ownership and permissions will be left at their default values if new values are not specified. Behavior when copying soft links from the collect directory is dependent on the behavior of the specified rcp command.
Parameters:
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rcpCommandList (Command as a list to be passed to util.executeCommand) - An rcp-compatible copy command to use for copying files
  • sourceDir (String representing a directory on disk) - Source directory
  • targetDir (String representing a directory on disk) - Target directory
  • ownership (Tuple of numeric ids (uid, gid)) - Owner and group that the copied files should have
  • permissions (UNIX permissions mode, specified in octal (i.e. 0640).) - Permissions that the staged files should have
Returns:
Number of files copied from the source directory to the target directory.
Raises:
  • ValueError - If source or target is not a directory or does not exist.
  • IOError - If there is an IO error copying the files.
Notes:
  • The returned count of copied files might be inaccurate if some of the copied files already existed in the staging directory prior to the copy taking place. We don't clear the staging directory first, because some extension might also be using it.
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • We don't have a good way of knowing exactly what files we copied down from the remote peer, unless we want to parse the output of the rcp command (ugh). We could change permissions on everything in the target directory, but that's kind of ugly too. Instead, we use Python's set functionality to figure out what files were added while we executed the rcp command. This isn't perfect - for instance, it's not correct if someone else is messing with the directory at the same time we're doing the remote copy - but it's about as good as we're going to get.
  • Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. We try to work around this by issuing IOError if we don't copy any files from the remote host.

_copyRemoteFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, ownership=None, permissions=None, overwrite=True)
Static Method

source code 
Copies a remote source file to a target file.
Parameters:
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rcpCommandList (Command as a list to be passed to util.executeCommand) - An rcp-compatible copy command to use for copying files
  • sourceFile (String representing a file on disk, as an absolute path) - Source file to copy
  • targetFile (String representing a file on disk, as an absolute path) - Target file to create
  • ownership (Tuple of numeric ids (uid, gid)) - Owner and group that the copied should have
  • permissions (UNIX permissions mode, specified in octal (i.e. 0640).) - Permissions that the staged files should have
  • overwrite (Boolean true/false.) - Indicates whether it's OK to overwrite the target file.
Raises:
  • IOError - If the target file already exists.
  • IOError - If there is an IO error copying the file
  • OSError - If there is an OS error changing permissions on the file
Notes:
  • Internally, we have to go through and escape any spaces in the source path with double-backslash, otherwise things get screwed up. It doesn't seem to be required in the target path. I hope this is portable to various different rcp methods, but I guess it might not be (all I have to test with is OpenSSH).
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • We will not overwrite a target file that exists when this method is invoked. If the target already exists, we'll raise an exception.
  • Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. We try to work around this by issuing IOError the target file does not exist when we're done.

_pushLocalFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, overwrite=True)
Static Method

source code 
Copies a local source file to a remote host.
Parameters:
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rcpCommandList (Command as a list to be passed to util.executeCommand) - An rcp-compatible copy command to use for copying files
  • sourceFile (String representing a file on disk, as an absolute path) - Source file to copy
  • targetFile (String representing a file on disk, as an absolute path) - Target file to create
  • overwrite (Boolean true/false.) - Indicates whether it's OK to overwrite the target file.
Raises:
  • IOError - If there is an IO error copying the file
  • OSError - If there is an OS error changing permissions on the file
Notes:
  • We will not overwrite a target file that exists when this method is invoked. If the target already exists, we'll raise an exception.
  • Internally, we have to go through and escape any spaces in the source and target paths with double-backslash, otherwise things get screwed up. I hope this is portable to various different rcp methods, but I guess it might not be (all I have to test with is OpenSSH).
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.

_setName(self, value)

source code 
Property target used to set the peer name. The value must be a non-empty string and cannot be None.
Raises:
  • ValueError - If the value is an empty string or None.

_setCollectDir(self, value)

source code 
Property target used to set the collect directory. The value must be an absolute path and cannot be None. It does not have to exist on disk at the time of assignment.
Raises:
  • ValueError - If the value is None or is not an absolute path.
  • ValueError - If the value cannot be encoded properly.

_setWorkingDir(self, value)

source code 
Property target used to set the working directory. The value must be an absolute path and cannot be None.
Raises:
  • ValueError - If the value is None or is not an absolute path.
  • ValueError - If the value cannot be encoded properly.

_setRemoteUser(self, value)

source code 
Property target used to set the remote user. The value must be a non-empty string and cannot be None.
Raises:
  • ValueError - If the value is an empty string or None.

_setLocalUser(self, value)

source code 
Property target used to set the local user. The value must be a non-empty string if it is not None.
Raises:
  • ValueError - If the value is an empty string.

_setRcpCommand(self, value)

source code 

Property target to set the rcp command.

The value must be a non-empty string or None. Its value is stored in the two forms: "raw" as provided by the client, and "parsed" into a list suitable for being passed to util.executeCommand via util.splitCommandLine.

However, all the caller will ever see via the property is the actual value they set (which includes seeing None, even if we translate that internally to DEF_RCP_COMMAND). Internally, we should always use self._rcpCommandList if we want the actual command list.
Raises:
  • ValueError - If the value is an empty string.

Property Details [hide private]

name

Name of the peer (a valid DNS hostname).
Get Method:
CedarBackup2.peer.RemotePeer._getName(self) - Property target used to get the peer name.
Set Method:
CedarBackup2.peer.RemotePeer._setName(self, value) - Property target used to set the peer name.

collectDir

Path to the peer's collect directory (an absolute local path).
Get Method:
CedarBackup2.peer.RemotePeer._getCollectDir(self) - Property target used to get the collect directory.
Set Method:
CedarBackup2.peer.RemotePeer._setCollectDir(self, value) - Property target used to set the collect directory.

remoteUser

Name of the Cedar Backup user on the remote peer.
Get Method:
CedarBackup2.peer.RemotePeer._getRemoteUser(self) - Property target used to get the remote user.
Set Method:
CedarBackup2.peer.RemotePeer._setRemoteUser(self, value) - Property target used to set the remote user.

rcpCommand

An rcp-compatible copy command to use for copying files.
Get Method:
CedarBackup2.peer.RemotePeer._getRcpCommand(self) - Property target used to get the rcp command.
Set Method:
CedarBackup2.peer.RemotePeer._setRcpCommand(self, value) - Property target to set the rcp command.

workingDir

Path to the peer's working directory (an absolute local path).
Get Method:
CedarBackup2.peer.RemotePeer._getWorkingDir(self) - Property target used to get the working directory.
Set Method:
CedarBackup2.peer.RemotePeer._setWorkingDir(self, value) - Property target used to set the working directory.

localUser

Name of the Cedar Backup user on the current host.
Get Method:
CedarBackup2.peer.RemotePeer._getLocalUser(self) - Property target used to get the local user.
Set Method:
CedarBackup2.peer.RemotePeer._setLocalUser(self, value) - Property target used to set the local user.