Bacula 1.33 User's Guide Chapter 17
Back
Utility Programs
Index
Index
Next
When Bacula crashes (Kaboom)

Testing Your Tape Drive With Bacula

This chapter is concerned with testing and configuring your tape drive to make sure that it will work properly with Bacula using the btape program.

Specifying the Configuration File

Starting with version 1.27, each of the btape program requires a valid Storage daemon configuration file (actually, the only part of the configuration file that btape needs is the Device resource definitions). This permits btape to find the configuration parameters for your archive device (generally a tape drive). By default, it reads bacula-sd.conf in the current directory, but you may specify a different configuration file using the -c option.

Specifying a Device Name For a Tape

btape device-name where the Volume can be found. In the case of a tape, this is the physical device name such as /dev/nst0 or /dev/rmt/0ubn depending on your system. For the program to work, it must find the identical name in the Device resource of the configuration file. See below for specifying Volume names.

Specifying a Device Name For a File

If you are attempting to read or write an archive file rather than a tape, the device-name should be the full path to the archive location including the filename. The filename (last part of the specification) will be stripped and used as the Volume name, and the path (first part before the filename) must have the same entry in the configuration file. So, the path is equivalent to the archive device name, and the filename is equivalent to the volume name.

btape

This program permits a number of elementary tape operations via a tty command interface. The test command, described below, can be very useful for testing older tape drive compatibility problems. Aside from initial testing of tape drive compatibility with Bacula, btape will be mostly used by developers writing new tape drivers.

btape can be dangerous to use with existing Bacula tapes because it will relabel a tape or write on the tape if so requested regardless that the tape may contain valuable data, so please be careful and use it only on blank tapes.

To work properly, btape needs to read the Storage daemon's configuration file. As a default, it will look for bacula-sd.conf in the current directory. If your configuration file is elsewhere, please use the -c option to specify where.

The physical device name must be specified on the command line, and that this same device name must be present in the Storage daemon's configuration file read by btape

Usage: btape [-c config_file] [-d debug_level] [device_name]
       -b <file>   specify bootstrap file
       -c <file>   set configuration file to file
       -d <nn>     set debug level to nn
       -s          turn off signals
       -v          be verbose
       -?          print this message.

Using btape to Verify your Tape Drive

An important reason for this program is to ensure that a Storage daemon configuration file is defined so that Bacula will correctly read and write tapes.

It is highly recommended that you run the test command before running your first Bacula job to ensure that the parameters you have defined for your storage device (tape drive) will permit Bacula to function properly. You only need to mount a blank tape, enter the command, and the output should be reasonably self explanatory. For example:

(ensure that Bacula is not running)
./btape -c /usr/bin/bacula/bacula-sd.conf /dev/nst0
The output will be:
Tape block granularity is 1024 bytes.
btape: btape.c:376 Using device: /dev/nst0
*
Enter the test command:
test
The output produced should be something similar to the following:

=== Append files test ===

This test is essential to Bacula.

I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2

btape: btape.c:387 Rewound /dev/nst0
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:387 Rewound /dev/nst0
btape: btape.c:693 Now moving to end of media.
btape: btape.c:427 Moved to end of media
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:387 Rewound /dev/nst0
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451136
End scanning the tape.
We should be in file 4. I am at file 4. This is correct!

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.


=== Write, backup, and re-read test ===

I'm going to write three records and two eof's
then backup over the eof's and re-read the last record.
Bacula does this after writing the last block on the
tape to verify that the block was written correctly.
It is not an *essential* feature ...

btape: btape.c:387 Rewound /dev/nst0
btape: btape.c:597 Wrote first record of 64412 bytes.
btape: btape.c:608 Wrote second record of 64412 bytes.
btape: btape.c:619 Wrote third record of 64412 bytes.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:631 Backspaced over two EOFs OK.
btape: btape.c:636 Backspace record OK.
btape: btape.c:658
Block re-read correct. Test succeeded!
=== End Write, backup, and re-read test ===


=== End Append files test ===
If you do not successfully complete the above test, please resolve the problem(s) before attempting to use Bacula. Depending on your tape drive, the test may recommend that you add certain records to your configuration. We strongly recommend that you do so and then re-run the above test to insure it works the first time.

Linux SCSI Tricks

You can find out what SCSI devices you have by doing:
cat /proc/scsi/scsi
For example, I get the fillowing:
Attached devices:
Host: scsi2 Channel: 00 Id: 01 Lun: 00
  Vendor: HP       Model: C5713A           Rev: H107
  Type:   Sequential-Access                ANSI SCSI revision: 02
Host: scsi2 Channel: 00 Id: 04 Lun: 00
  Vendor: SONY     Model: SDT-10000        Rev: 0110
  Type:   Sequential-Access                ANSI SCSI revision: 02
If you want to remove the SDT-10000 device, you can do so as root with:
echo "scsi remove-single-device 2 0 4 0">/proc/scsi/scsi
and you can put add it back with:
echo "scsi add-single-device 2 0 4 0">/proc/scsi/scsi
where the 2 0 4 0 are the Host, Channel, Id, and Lun as seen on the output from cat /proc/scsi/scsi. Note, the Channel must be specified as numeric.

Tips for Resolving Problems

Bacula Cannot Open the Device

If you get an error message such as:
dev open failed: dev.c:265 stored: unable to open
device /dev/nst0:> ERR=No such device or address
the first time you run a job, it is most likely due to the fact that you specified the incorrect device name on your Archive Device.

If Bacula works fine with your drive, then all off a sudden you get error messages similar to the one shown above, it is quite possible that your driver module is being removed because the kernel deems it idle. This is done via crontab with the use of rmmod -a. To fix the problem, you can remove this entry from crontab, or you can manually modprob your driver module (or add it to the local startup script). Thanks to Alan Brown for this tip.

Incorrect File Number

When Bacula moves to the end of the medium, it normally uses the ioctl(MTEOM) function. Then Bacula uses the ioctl(MTIOCGET) function to retrieve the current file position from the mt_fileno field. Some SCSI tape drivers will use a fast means of seeking to the end of the medium and in doing so, they will not know the current file position and hence return a -1. As a consequence, if you get "This is NOT correct!" in the positioning tests, this may be the cause. You must correct this condition in order for Bacula to work.

There are two possible solutions to the above problem of incorrect file number:

  • Figure out how to configure your SCSI driver to keep track of the file position during the MTEOM request. This is the preferred solution.
  • Modify the Device resource of your bacula-sd.conf file to include:
    Hardware End of File = no
    This will cause Bacula to use the MTFSF request to seek to the end of the medium, and Bacula will keep track of the file number itself.

Incorrect Number of Blocks

Bacula's preferred method of working with tape drives (sequential devices) is to run in variable block mode. All modern tape drives support this mode, but some older drives (in particular the QIC drives) as well as the ATAPI ide-scsi driver run only in fixed block mode.

Even in variable block mode, with the exception of the first record on the second or subsequent volume of a multi-volume backup, Bacula will write blocks of a fixed size. However, in reading a tape, Bacula will assume that for each read request, exactly one block from the tape will be transferred. This the most common way that tape drives work and is well supported by Bacula.

Drives that run in fixed block mode can cause serious problems for Bacula if the drive's block size does not correspond exactly to Bacula's block size. In fixed block size mode, drivers may transmit a partial block or multiple blocks for a single read request. From Bacula's point of view, this destroys the concept of tape blocks. In order for Bacula to run in fixed block mode, you must include the following records in the Storage daemon's Device resource definition:

Minimum Block Size = nnn
Maximum Block Size = nnn
where nnn must be the same for both records and must be identical to the driver's fixed block size.

We recommend that you avoid this configuration if at all possible. In any case, as of version 1.27, it is not at all clear that Bacula's handling of these fixed block drivers really works.

Ensuring that the Tape Modes Are Properly Set -- Linux Only

If you have a modern SCSI tape drive and you are having problems with the test command as noted above, it may be that some program has set one or more of the your SCSI driver's options to non-default values. For example, if your driver is set to work in SysV manner, Bacula will not work correctly because it expects BSD behavior. To reset your tape drive to the default values, you can try the following, but ONLY if you have a SCSI tape drive on a Linux system:
become super user
mt -f /dev/nst0 rewind
mt -f /dev/nst0 stoptions buffer-writes async-writes read-ahead
The above commands will clear all options and then set those specified. None of the specified options are required by Bacula, but a number of other options such as SysV behavior must not be set. On systems other than Linux, you will need to consult your mt man pages or documentation to figure out how to do the same thing. This should not really be necessary though -- for example, on both Linux and Solaris systems, the default tape driver options are compatible with Bacula.

You may also want to ensure that no prior program has set the default block size, as happened to one user, by explicitly turning it off with:

mt -f /dev/nst0 defblksize 0

If you would like to know what stoptions you have set before making any of the changes noted above, you can now view them on Linux systems, thanks to a tip provided by Willem Riede. Do the following:

become super user
mt -f /dev/nst0 stsetoptions 0
grep st0 /var/log/messages
and you will get output that looks something like the following:
kernel: st0: Mode 0 options: buffer writes: 1, async writes: 1, read ahead: 1
kernel: st0:    can bsr: 0, two FMs: 0, fast mteom: 0, auto lock: 0,
kernel: st0:    defs for wr: 0, no block limits: 0, partitions: 0, s2 log: 0
kernel: st0:    sysv: 0 nowait: 0
Note, I have chopped off the beginning of the line with the date and machine name for presentation purposes.

Checking and Setting Tape Hardware Compression

As far as I can tell, there is no way with the mt program to check if your tape hardware compression is turned on or off. You can, however, turn it on by using (on Linux):
become super user
mt -f /dev/nst0 defcompression 1
and of course, if you use a zero instead of the one at the end, you will turn it off.

You may also want to ensure that no prior program has set the default block size, as happened to one user, by explicitly turning it off with:

mt -f /dev/nst0 defblksize 0

If you have built the mtx program in the depkgs package, you can use tapeinfo to get quite a bit of information about your tape drive even if it is not an autochanger. For example on my DDS-4 drive, I get the following:

./tapeinfo -f /dev/sg0
Product Type: Tape Drive
Vendor ID: 'HP      '
Product ID: 'C5713A          '
Revision: 'H107'
Attached Changer: No
MinBlock:1
MaxBlock:16777215
SCSI ID: 5
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x26
BlockSize: 0             <==== check this
DataCompEnabled: yes     <==== check this
DataCompCapable: yes
DataDeCompEnabled: yes   <==== check this
CompType: 0x20
DeCompType: 0x0
Block Position: 1141
where the DataCompEnabled: yes means that tape hardware compression is turned on. You can see it turn on and off (yes/no) by using the mt commands given above. Also, this output will tell you if the BlockSize is non-zero and hence set for a particular block size. Bacula is not likely to work in such a situation because it will normally attempt to write blocks of 64,512 bytes, except the last block of the job which will generally be shorter. The first thing to try is setting the default block size to zero using the mt -f /dev/nst0 defblksize 0 command as shown above.

If your tape drive requires fixed block sizes (very unusual), you can use the following records:

Minimum Block Size = nnn
Maximum Block Size = nnn
in your Storage daemon's Device resource to force Bacula to write fixed size blocks (where you sent nnn to be the same for both of the above records)

Also, note that the Medium Type field of the output of tapeinfo reports Not Loaded, which is not correct. As a consequence, you should ignore that field as well as the Attached Changer field.

Tape Modes on FreeBSD

Normally Bacula should run with: mt -f /dev/nsa0 seteotmodel 2, and then according to what the btape test command returns, you will probably need to set the following:
Hardware End of Medium = no
BSF at EOM = yes
but you should start the test command without these records and then add them if it requests you to do so. Normally, you will also want mt -f /dev/nsa0 comp on to be set.

Please see the file platforms/freebsd/pthreads-fix.txt in the main Bacula directory concerning important information concerning compatibility of Bacula and your system.

Determining What Tape Drives and Autochangers You Have on FreeBSD

On FreeBSD, you can do a camcontrol devlist as root to determine what drives and autochangers you have. For example,
undef# camcontrol devlist
    at scbus0 target 2 lun 0 (pass0,sa0)
    at scbus0 target 4 lun 0 (pass1,sa1)
    at scbus0 target 4 lun 1 (pass2)
from the above, you can determine that there is a tape drive on /dev/sa0 and another on /dev/sa1 in addition since there is a second line for the drive on /dev/sa1>, you know can assume that it is the control device for the autochanger (i.e. /dev/pass2.

Using the OnStream driver on Linux Systems

Bacula version 1.33 (not 1.32x) is now working and ready for testing with the OnStream kernel osst driver version 0.9.14 or above. Osst is available from: http://sourceforge.net/projects/osst/.

To make Bacula work you must first load the new driver then, as root, do:

  mt -f /dev/nosst0 defblksize 32768
Also you must add the following to your Device resource in your Storage daemon's conf file:
 Minimum Block Size = 32768
 Maximum Block Size = 32768
Here is a Device specification provided by Michel Meyers that is known to work:
Device {
  Name = "Onstream DI-30"
  Media Type = "ADR-30"
  Archive Device = /dev/tape
  Minimum Block Size = 32768
  Maximum Block Size = 32768
  Hardware End of Medium = yes
  BSF at EOM = no
  Backward Space File = yes
  Fast Forward Space File = yes
  Two EOF = no
  AutomaticMount = yes
  AlwaysOpen = yes
  Removable Media = yes
}

Using btape to Simulate Bacula Filling a Tape

Because there are often problems with certain tape drives or systems when end of tape conditions occur, btape has a special command fill that causes it to write random data to a tape until the tape fills. It then writes at least one more Bacula block to a second tape. Finally, it reads back both tapes to ensure that the data has been written in a way that Bacula can recover it. Note, there is also a single tape option as noted below, which you should use rather than the two tape test. See below for more details.

This can be an extremely time consuming process (here is is about 6 hours), and you must have two blank tapes available. As this command is completely new, it is not well tested, and has considerable room for improvement, especially during the error checking in the read-back phase.

To begin this test, you enter the fill command and follow the instructions. There are two options: the simple single tape option and the multiple tape option. Please use only the simple single tape option because the multiple tape option still doesn't work totally correctly. If the single tape option does not succeed, you should correct the problem before using Bacula.

Tape Blocking Modes

SCSI tapes may either be written in variable or fixed block sizes. Newer drives support both modes, but some drives such as the QIC devices always use fixed block sizes. Bacula attempts to fill and write complete blocks (default 65K), so that in normal mode (variable block size), Bacula will always write blocks of the same size except the last block of a Job. If Bacula is configured to write fixed block sizes, it will pad the last block of the Job to the correct size. Bacula expects variable tape block size drives to behave as follows: Each write to the drive results in a single record being written to the tape. Each read returns a single record. If you request less byte than are in the record, only those number of bytes will be returned, but the entire logical record will have been read (the next read will retrieve the next record). Thus data from a single write is always returned in a single read, and sequentially written records are returned by sequential reads.

Bacula expects fixed block size tape drives to behave as follows: If a write length is greater than the physical block size of the drive, the write will be written as two blocks each of the fixed physical size. This a single write may become multiple physical records on the tape. (This is not a good situation). According to the documentation, one may never write an amount of data that is not the exact multiple of the blocksize (it is not specified if an error occurs or if the the last record is padded). When reading, it is my understanding that each read request reads one physical record from the tape. Due to the complications of fixed block size tape drives, you should avoid them if possible with Bacula, or you must be ABSOLUTELY certain that you use fixed block sizes within Bacula that correspond to the physical block size of the tape drive. This will ensure that Bacula has a one to one correspondence between what it writes and the physical record on the tape.

Please note that Bacula will not function correctly if it writes a block and that block is split into two or more physical records on the tape. Bacula assumes that each write causes a single record to be written, and that it can sequentially recover each of the blocks it has written by using the same number of sequential reads as it had written.



Back
Utility Programs
Index
Index
Next
When Bacula crashes (Kaboom)
Bacula 1.33 User's Guide
The Network Backup Solution
Copyright © 2000-2004
Kern Sibbald and John Walker