Home | Trees | Index | Help |
|
---|
Package CedarBackup2 :: Module action |
|
Provides implementation of various backup-related actions.
The command-line interface is mostly implemented in terms of the action-related functionality in this module. There is one process function for each high-level backup action (collect, stage, store, purge, rebuild). In turn, each of the action functions is mostly implemented in terms of lower-level functionality in the other Cedar Backup modules. This is mostly "glue" code.
All of the public action functions in this file implements the Cedar Backup Extension Architecture Interface, i.e. the same interface that extensions will implement. There's no particular reason it has to be this way, except that it seems more straightforward to do it this way.
The code is organized into three rough sections: general utility code, attribute "getter" functions, and public functions. Attribute getter function encode rules for getting the correct value for various attributes. For instance, what do we do when the device type is unset or if a collect dir doesn't have an ignore file set, etc. They are grouped roughly by the action that they are associated with. Other utility functions related to a single public function are grouped with that function (below it, typically).Author: Kenneth J. Pronovici <pronovic@ieee.org>
Function Summary | |
---|---|
Executes the collect backup action. | |
Executes the stage backup action. | |
Executes the store backup action. | |
Executes the purge backup action. | |
Executes the rebuild backup action. | |
Executes the validate action. | |
Checks that the indicated directory is OK. | |
Collects a configured collect directory. | |
Collects a configured collect file. | |
Runs a consistency check against media in the backup device. | |
Creates staging directories as required. | |
Converts English day name to numeric day of week as from time.localtime . | |
Execute the backup process for the indicated backup list. | |
Finds the correct daily staging directory to be written to disk. | |
Finds the set of directories to be included in a disc rebuild. | |
Gets the archive mode that should be used for a collect directory or file. | |
Gets the collect mode that should be used for a collect directory or file. | |
Gets the daily staging directory. | |
Gets the device type that should be used for storing. | |
Gets the digest path associated with a collect directory or file. | |
Gets exclusions (file and patterns) associated with a collect directory. | |
Gets the ignore file that should be used for a collect directory or file. | |
Return a list of LocalPeer objects based on
configuration. | |
Gets the media type that should be used for storing. | |
Gets the RCP command associated with a remote peer. | |
Return a list of RemotePeer objects based on
configuration. | |
Gets the remote user associated with a remote peer. | |
Gets the tarfile path (including correct extension) associated with a collect directory. | |
Gets a writer object based on current configuration. | |
Loads the indicated digest path from disk into a dictionary. | |
Execute runtime validations on collect configuration. | |
Execute runtime validations on extensions configuration. | |
Execute runtime validations on options configuration. | |
Execute runtime validations on purge configuration. | |
Execute runtime validations on reference configuration. | |
Execute runtime validations on stage configuration. | |
Execute runtime validations on store configuration. | |
Writes a collect indicator file into a target collect directory. | |
Writes the digest dictionary to the indicated digest path on disk. | |
Builds and writes an ISO image containing the indicated stage directories. | |
Writes a stage indicator file into the daily staging directory. | |
Writes a store indicator file into staging directories. | |
Returns a "normalized" path based on an absolute path. | |
Indicates whether "today" is the backup starting day per configuration. |
Variable Summary | |
---|---|
str |
COLLECT_INDICATOR = 'cback.collect'
|
str |
DIGEST_EXTENSION = 'sha'
|
str |
DIR_TIME_FORMAT = '%Y/%m/%d'
|
Logger |
logger = <logging.Logger instance at 0x4055b78c>
|
str |
PREFIX_TIME_FORMAT = '%Y/%m/%d'
|
int |
SECONDS_PER_DAY = 86400 |
str |
STAGE_INDICATOR = 'cback.stage'
|
str |
STORE_INDICATOR = 'cback.store'
|
Function Details |
---|
executeCollect(configPath, options, config)Executes the collect backup action.
|
executeStage(configPath, options, config)Executes the stage backup action.
|
executeStore(configPath, options, config)Executes the store backup action. Note that the rebuild action and the store action are very similar. The main difference is that while store only stores a single day's staging directory, the rebuild action operates on multiple staging directories.
|
executePurge(configPath, options, config)Executes the purge backup action. For each configured directory, we create a purge item list, remove from the list anything that's younger than the configured retain days value, and then purge from the filesystem what's left.
|
executeRebuild(configPath, options, config)Executes the rebuild backup action. This function exists mainly to recreate a disc that has been "trashed" due to media or hardware problems. Note that the "stage complete" indicator isn't checked for this action. Note that the rebuild action and the store action are very similar. The main difference is that while store only stores a single day's staging directory, the rebuild action operates on multiple staging directories.
|
executeValidate(configPath, options, config)Executes the validate action. This action validates each of the individual sections in the config file. This is a "runtime" validation. The config file itself is already valid in a structural sense, so what we check here that is that we can actually use the configuration without any problems. There's a separate validation function for each of the configuration sections. Each validation function returns a true/false indication for whether configuration was valid, and then logs any configuration problems it finds. This way, one pass over configuration indicates most or all of the obvious problems, rather than finding just one problem at a time. Any reported problems will be logged at the ERROR level normally, or at the INFO level if the quiet flag is enabled.
|
_checkDir(path, writable, logfunc, prefix)Checks that the indicated directory is OK. The path must exist, must be a directory, must be readable and executable, and must optionally be writable.
|
_collectDirectory(config, absolutePath, tarfilePath, collectMode, archiveMode, ignoreFile, resetDigest, digestPath, excludePaths, excludePatterns)Collects a configured collect directory. The indicated collect directory is collected into the indicated tarfile. For directories that are collected incrementally, we'll use the indicated digest path and pay attention to the reset digest flag (basically, the reset digest flag ignores any existing digest, but a new digest is always rewritten). The caller must decide what the collect and archive modes are, since they can be on both the collect configuration and the collect directory itself.
|
_collectFile(config, absolutePath, tarfilePath, collectMode, archiveMode, resetDigest, digestPath)Collects a configured collect file. The indicated collect file is collected into the indicated tarfile. For files that are collected incrementally, we'll use the indicated digest path and pay attention to the reset digest flag (basically, the reset digest flag ignores any existing digest, but a new digest is always rewritten). The caller must decide what the collect and archive modes are, since they can be on both the collect configuration and the collect file itself.
|
_consistencyCheck(config, stagingDirs)Runs a consistency check against media in the backup device. It seems that sometimes, it's possible to create a corrupted multisession disc (i.e. one that cannot be read) although no errors were encountered while writing the disc. This consistency check makes sure that the data read from disc matches the data that was used to create the disc. The function mounts the device at a temporary mount point in the
working directory, and then compares the indicated staging directories
in the staging directory and on the media. The comparison is done via
functionality in info priority.
|
_createStagingDirs(config, dailyDir, peers)Creates staging directories as required. The main staging directory is the passed in daily directory, something likestaging/2002/05/23 . Then, individual peers
get their own directories, i.e.
staging/2002/05/23/host .
|
_deriveDayOfWeek(dayName)Converts English day name to numeric day of week as from
monday would be converted to the
number 0 .
|
_executeBackup(config, backupList, absolutePath, tarfilePath, collectMode, archiveMode, resetDigest, digestPath)Execute the backup process for the indicated backup list. This function exists mainly to consolidate functionality between the
|
_findCorrectDailyDir(options, config)Finds the correct daily staging directory to be written to disk. In Cedar Backup v1.0, we assumed that the correct staging directory matched the current date. However, that has problems. In particular, it breaks down if collect is on one side of midnite and stage is on the other, or if certain processes span midnite. For v2.0, I'm trying to be smarter. I'll first check the current
day. If that directory is found, it's good enough. If it's not found,
I'll look for a valid directory from the day before or day after
which has not yet been staged, according to the stage indicator
file. The first one I find, I'll use. If I use a directory other
than for the current day and
options.full flag is set, then the special "span
midnite" logic will be disabled and any existing store indicator
will be ignored. I did this because I think that most users who run
cback --full store twice in a row expect the command to
generate two identical discs. With the other rule in place, running
that command twice in a row could result in an error ("no unstored
directory exists") or could even cause a completely unexpected
directory to be written to disc (if some previous day's contents had
not yet been written).
|
_findRebuildDirs(config)Finds the set of directories to be included in a disc rebuild. A the rebuild action is supposed to recreate the "last week's" disc. This won't always be possible if some of the staging directories are missing. However, the general procedure is to look back into the past no further than the previous "starting day of week", and then work forward from there trying to find all of the staging directories between then and now that still exist and have a stage indicator.
|
_getArchiveMode(config, item)Gets the archive mode that should be used for a collect directory or file. If possible, use the one on the file or directory, otherwise take from collect section.
|
_getCollectMode(config, item)Gets the collect mode that should be used for a collect directory or file. If possible, use the one on the file or directory, otherwise take from collect section.
|
_getDailyDir(config)Gets the daily staging directory. This is just a directory in the formstaging/YYYY/MM/DD , i.e. staging/2000/10/07 ,
except it will be an absolute path based on
config.stage.targetDir .
|
_getDeviceType(config)Gets the device type that should be used for storing. Use the configured device type if notNone , otherwise
use config.DEFAULT_DEVICE_TYPE .
|
_getDigestPath(config, item)Gets the digest path associated with a collect directory or file.
|
_getExclusions(config, collectDir)Gets exclusions (file and patterns) associated with a collect directory. The returned files value is a list of absolute paths to be excluded from the backup for a given directory. It is derived from the collect configuration absolute exclude paths and the collect directory's absolute and relative exclude paths. The returned patterns value is a list of patterns to be excluded from the backup for a given directory. It is derived from the list of patterns from the collect configuration and from the collect directory itself.
|
_getIgnoreFile(config, item)Gets the ignore file that should be used for a collect directory or file. If possible, use the one on the file or directory, otherwise take from collect section.
|
_getLocalPeers(config)Return a list ofLocalPeer objects based on
configuration.
|
_getMediaType(config)Gets the media type that should be used for storing. Use the configured media type if not writer.py , one of
MEDIA_CDR_74 , MEDIA_CDRW_74 ,
MEDIA_CDR_80 or MEDIA_CDRW_80 .
|
_getRcpCommand(config, remotePeer)Gets the RCP command associated with a remote peer. Use peer's if possible, otherwise take from options section.
|
_getRemotePeers(config)Return a list ofRemotePeer objects based on
configuration.
|
_getRemoteUser(config, remotePeer)Gets the remote user associated with a remote peer. Use peer's if possible, otherwise take from options section.
|
_getTarfilePath(config, item, archiveMode)Gets the tarfile path (including correct extension) associated with a collect directory.
|
_getWriter(config)Gets a writer object based on current configuration. This function creates and returns a writer based on configuration. This is done to abstract action methods from knowing what kind of writer is in use. Since all writers implement the same interface, there's no need for actions to care which one they're working with. Right now, only the
|
_loadDigest(digestPath)Loads the indicated digest path from disk into a dictionary. If we can't load the digest successfully (either because it doesn't exist or for some other reason), then an empty dictionary will be returned - but the condition will be logged.
|
_validateCollect(config, logfunc)Execute runtime validations on collect configuration. The following validations are enforced:
|
_validateExtensions(config, logfunc)Execute runtime validations on extensions configuration. The following validations are enforced:
|
_validateOptions(config, logfunc)Execute runtime validations on options configuration. The following validations are enforced:
|
_validatePurge(config, logfunc)Execute runtime validations on purge configuration. The following validations are enforced:
|
_validateReference(config, logfunc)Execute runtime validations on reference configuration. We only validate that reference configuration exists at all.
|
_validateStage(config, logfunc)Execute runtime validations on stage configuration. The following validations are enforced:
|
_validateStore(config, logfunc)Execute runtime validations on store configuration. The following validations are enforced:
|
_writeCollectIndicator(config)Writes a collect indicator file into a target collect directory.
|
_writeDigest(config, digest, digestPath)Writes the digest dictionary to the indicated digest path on disk. If we can't write the digest successfully for any reason, we'll log the condition but won't throw an exception.
|
_writeImage(config, entireDisc, stagingDirs)Builds and writes an ISO image containing the indicated stage directories. The generated image will contain each of the staging directories listed instagingDirs . The directories will be placed into
the image at the root by date, so staging directory
/opt/stage/2005/02/10 will be placed into the disc at
/2005/02/10 .
|
_writeStageIndicator(config, dailyDir)Writes a stage indicator file into the daily staging directory. Note that there is a stage indicator on each peer (to indicate that a collect directory has been staged) and in the daily staging directory itself (to indicate that the staging directory has been utilized). This just deals with the daily staging directory.
|
_writeStoreIndicator(config, stagingDirs)Writes a store indicator file into staging directories. The store indicator is written into each of the staging directories when either a store or rebuild action has written the staging directory to disc.
|
buildNormalizedPath(absPath)Returns a "normalized" path based on an absolute path. A "normalized" path has its leading'/' or
'.' characters removed, and then converts all remaining
whitespace and '/' characters to the '_'
character. As a special case, the absolute path / will be
normalized to just '-' .
|
isStartOfWeek(startingDay)Indicates whether "today" is the backup starting day per configuration. If the current day's English name matches the indicated starting day, then today is a starting day.
|
Variable Details |
---|
COLLECT_INDICATOR
|
DIGEST_EXTENSION
|
DIR_TIME_FORMAT
|
logger
|
PREFIX_TIME_FORMAT
|
SECONDS_PER_DAY
|
STAGE_INDICATOR
|
STORE_INDICATOR
|
Home | Trees | Index | Help |
|
---|
Generated by Epydoc 2.1 on Mon Dec 18 22:53:31 2006 | http://epydoc.sf.net |