1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39 """
40 Provides utilities related to image writers.
41 @author: Kenneth J. Pronovici <pronovic@ieee.org>
42 """
43
44
45
46
47
48
49
50 import os
51 import re
52 import logging
53
54
55 from CedarBackup2.filesystem import FilesystemList
56 from CedarBackup2.knapsack import worstFit
57 from CedarBackup2.util import resolveCommand, executeCommand
58 from CedarBackup2.util import convertSize, UNIT_BYTES, UNIT_SECTORS, encodePath
59
60
61
62
63
64
65 logger = logging.getLogger("CedarBackup2.log.writers.util")
66
67 MKISOFS_COMMAND = [ "mkisofs", ]
68 VOLNAME_COMMAND = [ "volname", ]
69
70
71
72
73
74
75
76
77
78
80 """
81 Validates a configured device.
82 The device must be an absolute path, must exist, and must be writable.
83 The unittest flag turns off validation of the device on disk.
84 @param device: Filesystem device path.
85 @param unittest: Indicates whether we're unit testing.
86 @return: Device as a string, for instance C{"/dev/cdrw"}
87 @raise ValueError: If the device value is invalid.
88 @raise ValueError: If some path cannot be encoded properly.
89 """
90 if device is None:
91 raise ValueError("Device must be filled in.")
92 device = encodePath(device)
93 if not os.path.isabs(device):
94 raise ValueError("Backup device must be an absolute path.")
95 if not unittest and not os.path.exists(device):
96 raise ValueError("Backup device must exist on disk.")
97 if not unittest and not os.access(device, os.W_OK):
98 raise ValueError("Backup device is not writable by the current user.")
99 return device
100
101
102
103
104
105
107 """
108 Validates a SCSI id string.
109 SCSI id must be a string in the form C{[<method>:]scsibus,target,lun}.
110 For Mac OS X (Darwin), we also accept the form C{IO.*Services[/N]}.
111 @note: For consistency, if C{None} is passed in, C{None} will be returned.
112 @param scsiId: SCSI id for the device.
113 @return: SCSI id as a string, for instance C{"ATA:1,0,0"}
114 @raise ValueError: If the SCSI id string is invalid.
115 """
116 if scsiId is not None:
117 pattern = re.compile(r"^\s*(.*:)?\s*[0-9][0-9]*\s*,\s*[0-9][0-9]*\s*,\s*[0-9][0-9]*\s*$")
118 if not pattern.search(scsiId):
119 pattern = re.compile(r"^\s*IO.*Services(\/[0-9][0-9]*)?\s*$")
120 if not pattern.search(scsiId):
121 raise ValueError("SCSI id is not in a valid form.")
122 return scsiId
123
124
125
126
127
128
130 """
131 Validates a drive speed value.
132 Drive speed must be an integer which is >= 1.
133 @note: For consistency, if C{None} is passed in, C{None} will be returned.
134 @param driveSpeed: Speed at which the drive writes.
135 @return: Drive speed as an integer
136 @raise ValueError: If the drive speed value is invalid.
137 """
138 if driveSpeed is None:
139 return None
140 try:
141 intSpeed = int(driveSpeed)
142 except TypeError:
143 raise ValueError("Drive speed must be an integer >= 1.")
144 if intSpeed < 1:
145 raise ValueError("Drive speed must an integer >= 1.")
146 return intSpeed
147
148
149
150
151
152
153
154
155
156
172
173
174
175
176
177
179
180
181
182
183
184 """
185 Represents an ISO filesystem image.
186
187 Summary
188 =======
189
190 This object represents an ISO 9660 filesystem image. It is implemented
191 in terms of the C{mkisofs} program, which has been ported to many
192 operating systems and platforms. A "sensible subset" of the C{mkisofs}
193 functionality is made available through the public interface, allowing
194 callers to set a variety of basic options such as publisher id,
195 application id, etc. as well as specify exactly which files and
196 directories they want included in their image.
197
198 By default, the image is created using the Rock Ridge protocol (using the
199 C{-r} option to C{mkisofs}) because Rock Ridge discs are generally more
200 useful on UN*X filesystems than standard ISO 9660 images. However,
201 callers can fall back to the default C{mkisofs} functionality by setting
202 the C{useRockRidge} instance variable to C{False}. Note, however, that
203 this option is not well-tested.
204
205 Where Files and Directories are Placed in the Image
206 ===================================================
207
208 Although this class is implemented in terms of the C{mkisofs} program,
209 its standard "image contents" semantics are slightly different than the original
210 C{mkisofs} semantics. The difference is that files and directories are
211 added to the image with some additional information about their source
212 directory kept intact.
213
214 As an example, suppose you add the file C{/etc/profile} to your image and
215 you do not configure a graft point. The file C{/profile} will be created
216 in the image. The behavior for directories is similar. For instance,
217 suppose that you add C{/etc/X11} to the image and do not configure a
218 graft point. In this case, the directory C{/X11} will be created in the
219 image, even if the original C{/etc/X11} directory is empty. I{This
220 behavior differs from the standard C{mkisofs} behavior!}
221
222 If a graft point is configured, it will be used to modify the point at
223 which a file or directory is added into an image. Using the examples
224 from above, let's assume you set a graft point of C{base} when adding
225 C{/etc/profile} and C{/etc/X11} to your image. In this case, the file
226 C{/base/profile} and the directory C{/base/X11} would be added to the
227 image.
228
229 I feel that this behavior is more consistent than the original C{mkisofs}
230 behavior. However, to be fair, it is not quite as flexible, and some
231 users might not like it. For this reason, the C{contentsOnly} parameter
232 to the L{addEntry} method can be used to revert to the original behavior
233 if desired.
234
235 @sort: __init__, addEntry, getEstimatedSize, _getEstimatedSize, writeImage,
236 _buildDirEntries _buildGeneralArgs, _buildSizeArgs, _buildWriteArgs,
237 device, boundaries, graftPoint, useRockRidge, applicationId,
238 biblioFile, publisherId, preparerId, volumeId
239 """
240
241
242
243
244
245 - def __init__(self, device=None, boundaries=None, graftPoint=None):
246 """
247 Initializes an empty ISO image object.
248
249 Only the most commonly-used configuration items can be set using this
250 constructor. If you have a need to change the others, do so immediately
251 after creating your object.
252
253 The device and boundaries values are both required in order to write
254 multisession discs. If either is missing or C{None}, a multisession disc
255 will not be written. The boundaries tuple is in terms of ISO sectors, as
256 built by an image writer class and returned in a L{writer.MediaCapacity}
257 object.
258
259 @param device: Name of the device that the image will be written to
260 @type device: Either be a filesystem path or a SCSI address
261
262 @param boundaries: Session boundaries as required by C{mkisofs}
263 @type boundaries: Tuple C{(last_sess_start,next_sess_start)} as returned from C{cdrecord -msinfo}, or C{None}
264
265 @param graftPoint: Default graft point for this page.
266 @type graftPoint: String representing a graft point path (see L{addEntry}).
267 """
268 self._device = None
269 self._boundaries = None
270 self._graftPoint = None
271 self._useRockRidge = True
272 self._applicationId = None
273 self._biblioFile = None
274 self._publisherId = None
275 self._preparerId = None
276 self._volumeId = None
277 self.entries = { }
278 self.device = device
279 self.boundaries = boundaries
280 self.graftPoint = graftPoint
281 self.useRockRidge = True
282 self.applicationId = None
283 self.biblioFile = None
284 self.publisherId = None
285 self.preparerId = None
286 self.volumeId = None
287 logger.debug("Created new ISO image object.")
288
289
290
291
292
293
295 """
296 Property target used to set the device value.
297 If not C{None}, the value can be either an absolute path or a SCSI id.
298 @raise ValueError: If the value is not valid
299 """
300 try:
301 if value is None:
302 self._device = None
303 else:
304 if os.path.isabs(value):
305 self._device = value
306 else:
307 self._device = validateScsiId(value)
308 except ValueError:
309 raise ValueError("Device must either be an absolute path or a valid SCSI id.")
310
312 """
313 Property target used to get the device value.
314 """
315 return self._device
316
318 """
319 Property target used to set the boundaries tuple.
320 If not C{None}, the value must be a tuple of two integers.
321 @raise ValueError: If the tuple values are not integers.
322 @raise IndexError: If the tuple does not contain enough elements.
323 """
324 if value is None:
325 self._boundaries = None
326 else:
327 self._boundaries = (int(value[0]), int(value[1]))
328
330 """
331 Property target used to get the boundaries value.
332 """
333 return self._boundaries
334
336 """
337 Property target used to set the graft point.
338 The value must be a non-empty string if it is not C{None}.
339 @raise ValueError: If the value is an empty string.
340 """
341 if value is not None:
342 if len(value) < 1:
343 raise ValueError("The graft point must be a non-empty string.")
344 self._graftPoint = value
345
347 """
348 Property target used to get the graft point.
349 """
350 return self._graftPoint
351
353 """
354 Property target used to set the use RockRidge flag.
355 No validations, but we normalize the value to C{True} or C{False}.
356 """
357 if value:
358 self._useRockRidge = True
359 else:
360 self._useRockRidge = False
361
363 """
364 Property target used to get the use RockRidge flag.
365 """
366 return self._useRockRidge
367
369 """
370 Property target used to set the application id.
371 The value must be a non-empty string if it is not C{None}.
372 @raise ValueError: If the value is an empty string.
373 """
374 if value is not None:
375 if len(value) < 1:
376 raise ValueError("The application id must be a non-empty string.")
377 self._applicationId = value
378
380 """
381 Property target used to get the application id.
382 """
383 return self._applicationId
384
386 """
387 Property target used to set the biblio file.
388 The value must be a non-empty string if it is not C{None}.
389 @raise ValueError: If the value is an empty string.
390 """
391 if value is not None:
392 if len(value) < 1:
393 raise ValueError("The biblio file must be a non-empty string.")
394 self._biblioFile = value
395
397 """
398 Property target used to get the biblio file.
399 """
400 return self._biblioFile
401
403 """
404 Property target used to set the publisher id.
405 The value must be a non-empty string if it is not C{None}.
406 @raise ValueError: If the value is an empty string.
407 """
408 if value is not None:
409 if len(value) < 1:
410 raise ValueError("The publisher id must be a non-empty string.")
411 self._publisherId = value
412
414 """
415 Property target used to get the publisher id.
416 """
417 return self._publisherId
418
420 """
421 Property target used to set the preparer id.
422 The value must be a non-empty string if it is not C{None}.
423 @raise ValueError: If the value is an empty string.
424 """
425 if value is not None:
426 if len(value) < 1:
427 raise ValueError("The preparer id must be a non-empty string.")
428 self._preparerId = value
429
431 """
432 Property target used to get the preparer id.
433 """
434 return self._preparerId
435
437 """
438 Property target used to set the volume id.
439 The value must be a non-empty string if it is not C{None}.
440 @raise ValueError: If the value is an empty string.
441 """
442 if value is not None:
443 if len(value) < 1:
444 raise ValueError("The volume id must be a non-empty string.")
445 self._volumeId = value
446
448 """
449 Property target used to get the volume id.
450 """
451 return self._volumeId
452
453 device = property(_getDevice, _setDevice, None, "Device that image will be written to (device path or SCSI id).")
454 boundaries = property(_getBoundaries, _setBoundaries, None, "Session boundaries as required by C{mkisofs}.")
455 graftPoint = property(_getGraftPoint, _setGraftPoint, None, "Default image-wide graft point (see L{addEntry} for details).")
456 useRockRidge = property(_getUseRockRidge, _setUseRockRidge, None, "Indicates whether to use RockRidge (default is C{True}).")
457 applicationId = property(_getApplicationId, _setApplicationId, None, "Optionally specifies the ISO header application id value.")
458 biblioFile = property(_getBiblioFile, _setBiblioFile, None, "Optionally specifies the ISO bibliographic file name.")
459 publisherId = property(_getPublisherId, _setPublisherId, None, "Optionally specifies the ISO header publisher id value.")
460 preparerId = property(_getPreparerId, _setPreparerId, None, "Optionally specifies the ISO header preparer id value.")
461 volumeId = property(_getVolumeId, _setVolumeId, None, "Optionally specifies the ISO header volume id value.")
462
463
464
465
466
467
468 - def addEntry(self, path, graftPoint=None, override=False, contentsOnly=False):
469 """
470 Adds an individual file or directory into the ISO image.
471
472 The path must exist and must be a file or a directory. By default, the
473 entry will be placed into the image at the root directory, but this
474 behavior can be overridden using the C{graftPoint} parameter or instance
475 variable.
476
477 You can use the C{contentsOnly} behavior to revert to the "original"
478 C{mkisofs} behavior for adding directories, which is to add only the
479 items within the directory, and not the directory itself.
480
481 @note: Things get I{odd} if you try to add a directory to an image that
482 will be written to a multisession disc, and the same directory already
483 exists in an earlier session on that disc. Not all of the data gets
484 written. You really wouldn't want to do this anyway, I guess.
485
486 @note: An exception will be thrown if the path has already been added to
487 the image, unless the C{override} parameter is set to C{True}.
488
489 @note: The method C{graftPoints} parameter overrides the object-wide
490 instance variable. If neither the method parameter or object-wide value
491 is set, the path will be written at the image root. The graft point
492 behavior is determined by the value which is in effect I{at the time this
493 method is called}, so you I{must} set the object-wide value before
494 calling this method for the first time, or your image may not be
495 consistent.
496
497 @note: You I{cannot} use the local C{graftPoint} parameter to "turn off"
498 an object-wide instance variable by setting it to C{None}. Python's
499 default argument functionality buys us a lot, but it can't make this
500 method psychic. :)
501
502 @param path: File or directory to be added to the image
503 @type path: String representing a path on disk
504
505 @param graftPoint: Graft point to be used when adding this entry
506 @type graftPoint: String representing a graft point path, as described above
507
508 @param override: Override an existing entry with the same path.
509 @type override: Boolean true/false
510
511 @param contentsOnly: Add directory contents only (standard C{mkisofs} behavior).
512 @type contentsOnly: Boolean true/false
513
514 @raise ValueError: If path is not a file or directory, or does not exist.
515 @raise ValueError: If the path has already been added, and override is not set.
516 @raise ValueError: If a path cannot be encoded properly.
517 """
518 path = encodePath(path)
519 if not override:
520 if path in self.entries.keys():
521 raise ValueError("Path has already been added to the image.")
522 if os.path.islink(path):
523 raise ValueError("Path must not be a link.")
524 if os.path.isdir(path):
525 if graftPoint is not None:
526 if contentsOnly:
527 self.entries[path] = graftPoint
528 else:
529 self.entries[path] = os.path.join(graftPoint, os.path.basename(path))
530 elif self.graftPoint is not None:
531 if contentsOnly:
532 self.entries[path] = self.graftPoint
533 else:
534 self.entries[path] = os.path.join(self.graftPoint, os.path.basename(path))
535 else:
536 if contentsOnly:
537 self.entries[path] = None
538 else:
539 self.entries[path] = os.path.basename(path)
540 elif os.path.isfile(path):
541 if graftPoint is not None:
542 self.entries[path] = graftPoint
543 elif self.graftPoint is not None:
544 self.entries[path] = self.graftPoint
545 else:
546 self.entries[path] = None
547 else:
548 raise ValueError("Path must be a file or a directory.")
549
551 """
552 Returns the estimated size (in bytes) of the ISO image.
553
554 This is implemented via the C{-print-size} option to C{mkisofs}, so it
555 might take a bit of time to execute. However, the result is as accurate
556 as we can get, since it takes into account all of the ISO overhead, the
557 true cost of directories in the structure, etc, etc.
558
559 @return: Estimated size of the image, in bytes.
560
561 @raise IOError: If there is a problem calling C{mkisofs}.
562 @raise ValueError: If there are no filesystem entries in the image
563 """
564 if len(self.entries.keys()) == 0:
565 raise ValueError("Image does not contain any entries.")
566 return self._getEstimatedSize(self.entries)
567
569 """
570 Returns the estimated size (in bytes) for the passed-in entries dictionary.
571 @return: Estimated size of the image, in bytes.
572 @raise IOError: If there is a problem calling C{mkisofs}.
573 """
574 args = self._buildSizeArgs(entries)
575 command = resolveCommand(MKISOFS_COMMAND)
576 (result, output) = executeCommand(command, args, returnOutput=True, ignoreStderr=True)
577 if result != 0:
578 raise IOError("Error (%d) executing mkisofs command to estimate size." % result)
579 if len(output) != 1:
580 raise IOError("Unable to parse mkisofs output.")
581 try:
582 sectors = float(output[0])
583 size = convertSize(sectors, UNIT_SECTORS, UNIT_BYTES)
584 return size
585 except:
586 raise IOError("Unable to parse mkisofs output.")
587
589 """
590 Writes this image to disk using the image path.
591
592 @param imagePath: Path to write image out as
593 @type imagePath: String representing a path on disk
594
595 @raise IOError: If there is an error writing the image to disk.
596 @raise ValueError: If there are no filesystem entries in the image
597 @raise ValueError: If a path cannot be encoded properly.
598 """
599 imagePath = encodePath(imagePath)
600 if len(self.entries.keys()) == 0:
601 raise ValueError("Image does not contain any entries.")
602 args = self._buildWriteArgs(self.entries, imagePath)
603 command = resolveCommand(MKISOFS_COMMAND)
604 (result, output) = executeCommand(command, args, returnOutput=False)
605 if result != 0:
606 raise IOError("Error (%d) executing mkisofs command to build image." % result)
607
608
609
610
611
612
614 """
615 Uses an entries dictionary to build a list of directory locations for use
616 by C{mkisofs}.
617
618 We build a list of entries that can be passed to C{mkisofs}. Each entry is
619 either raw (if no graft point was configured) or in graft-point form as
620 described above (if a graft point was configured). The dictionary keys
621 are the path names, and the values are the graft points, if any.
622
623 @param entries: Dictionary of image entries (i.e. self.entries)
624
625 @return: List of directory locations for use by C{mkisofs}
626 """
627 dirEntries = []
628 for key in entries.keys():
629 if entries[key] is None:
630 dirEntries.append(key)
631 else:
632 dirEntries.append("%s/=%s" % (entries[key].strip("/"), key))
633 return dirEntries
634 _buildDirEntries = staticmethod(_buildDirEntries)
635
637 """
638 Builds a list of general arguments to be passed to a C{mkisofs} command.
639
640 The various instance variables (C{applicationId}, etc.) are filled into
641 the list of arguments if they are set.
642 By default, we will build a RockRidge disc. If you decide to change
643 this, think hard about whether you know what you're doing. This option
644 is not well-tested.
645
646 @return: List suitable for passing to L{util.executeCommand} as C{args}.
647 """
648 args = []
649 if self.applicationId is not None:
650 args.append("-A")
651 args.append(self.applicationId)
652 if self.biblioFile is not None:
653 args.append("-biblio")
654 args.append(self.biblioFile)
655 if self.publisherId is not None:
656 args.append("-publisher")
657 args.append(self.publisherId)
658 if self.preparerId is not None:
659 args.append("-p")
660 args.append(self.preparerId)
661 if self.volumeId is not None:
662 args.append("-V")
663 args.append(self.volumeId)
664 return args
665
667 """
668 Builds a list of arguments to be passed to a C{mkisofs} command.
669
670 The various instance variables (C{applicationId}, etc.) are filled into
671 the list of arguments if they are set. The command will be built to just
672 return size output (a simple count of sectors via the C{-print-size} option),
673 rather than an image file on disk.
674
675 By default, we will build a RockRidge disc. If you decide to change
676 this, think hard about whether you know what you're doing. This option
677 is not well-tested.
678
679 @param entries: Dictionary of image entries (i.e. self.entries)
680
681 @return: List suitable for passing to L{util.executeCommand} as C{args}.
682 """
683 args = self._buildGeneralArgs()
684 args.append("-print-size")
685 args.append("-graft-points")
686 if self.useRockRidge:
687 args.append("-r")
688 if self.device is not None and self.boundaries is not None:
689 args.append("-C")
690 args.append("%d,%d" % (self.boundaries[0], self.boundaries[1]))
691 args.append("-M")
692 args.append(self.device)
693 args.extend(self._buildDirEntries(entries))
694 return args
695
697 """
698 Builds a list of arguments to be passed to a C{mkisofs} command.
699
700 The various instance variables (C{applicationId}, etc.) are filled into
701 the list of arguments if they are set. The command will be built to write
702 an image to disk.
703
704 By default, we will build a RockRidge disc. If you decide to change
705 this, think hard about whether you know what you're doing. This option
706 is not well-tested.
707
708 @param entries: Dictionary of image entries (i.e. self.entries)
709
710 @param imagePath: Path to write image out as
711 @type imagePath: String representing a path on disk
712
713 @return: List suitable for passing to L{util.executeCommand} as C{args}.
714 """
715 args = self._buildGeneralArgs()
716 args.append("-graft-points")
717 if self.useRockRidge:
718 args.append("-r")
719 args.append("-o")
720 args.append(imagePath)
721 if self.device is not None and self.boundaries is not None:
722 args.append("-C")
723 args.append("%d,%d" % (self.boundaries[0], self.boundaries[1]))
724 args.append("-M")
725 args.append(self.device)
726 args.extend(self._buildDirEntries(entries))
727 return args
728