Chapter 4 Programming Guidelines
4.1 CFITSIO Definitions
Any program that uses the CFITSIO interface must include the fitsio.h
header file with the statement
#include "fitsio.h"
This header file contains the prototypes for all the CFITSIO user
interface routines as well as the definitions of various constants used
in the interface. It also defines a C structure of type `fitsfile'
that is used by CFITSIO to store the relevant parameters that define
the format of a particular FITS file. Application programs must define
a pointer to this structure for each FITS file that is to be opened.
This structure is initialized (i.e., memory is allocated for the
structure) when the FITS file is first opened or created with the
fits_open_file or fits_create_file routines. This fitsfile pointer
is then passed as the first argument to every other CFITSIO routine
that operates on the FITS file. Application programs must not directly
read or write elements in this fitsfile structure because the
definition of the structure may change in future versions of CFITSIO.
A number of symbolic constants are also defined in fitsio.h for the
convenience of application programmers. Use of these symbolic
constants rather than the actual numeric value will help to make the
source code more readable and easier for others to understand.
String Lengths, for use when allocating character arrays:
#define FLEN_FILENAME 1025 /* max length of a filename */
#define FLEN_KEYWORD 72 /* max length of a keyword */
#define FLEN_CARD 81 /* max length of a FITS header card */
#define FLEN_VALUE 71 /* max length of a keyword value string */
#define FLEN_COMMENT 73 /* max length of a keyword comment string */
#define FLEN_ERRMSG 81 /* max length of a CFITSIO error message */
#define FLEN_STATUS 31 /* max length of a CFITSIO status text string */
Note that FLEN_KEYWORD is longer than the nominal 8-character keyword
name length because the HIERARCH convention supports longer keyword names.
Access modes when opening a FITS file:
#define READONLY 0
#define READWRITE 1
BITPIX data type code values for FITS images:
#define BYTE_IMG 8 /* 8-bit unsigned integers */
#define SHORT_IMG 16 /* 16-bit signed integers */
#define LONG_IMG 32 /* 32-bit signed integers */
#define FLOAT_IMG -32 /* 32-bit single precision floating point */
#define DOUBLE_IMG -64 /* 64-bit double precision floating point */
The following 4 data type codes are also supported by CFITSIO:
#define LONGLONG_IMG 64 /* 64-bit long signed integers */
#define SBYTE_IMG 10 /* 8-bit signed integers, equivalent to */
/* BITPIX = 8, BSCALE = 1, BZERO = -128 */
#define USHORT_IMG 20 /* 16-bit unsigned integers, equivalent to */
/* BITPIX = 16, BSCALE = 1, BZERO = 32768 */
#define ULONG_IMG 40 /* 32-bit unsigned integers, equivalent to */
/* BITPIX = 32, BSCALE = 1, BZERO = 2147483648 */
Codes for the data type of binary table columns and/or for the
data type of variables when reading or writing keywords or data:
DATATYPE TFORM CODE
#define TBIT 1 /* 'X' */
#define TBYTE 11 /* 8-bit unsigned byte, 'B' */
#define TLOGICAL 14 /* logicals (int for keywords */
/* and char for table cols 'L' */
#define TSTRING 16 /* ASCII string, 'A' */
#define TSHORT 21 /* signed short, 'I' */
#define TINT32BIT 41 /* signed 32-bit int, 'J' */
#define TLONG 41 /* signed long, */
#define TFLOAT 42 /* single precision float, 'E' */
#define TDOUBLE 82 /* double precision float, 'D' */
#define TCOMPLEX 83 /* complex (pair of floats) 'C' */
#define TDBLCOMPLEX 163 /* double complex (2 doubles) 'M' */
The following data type codes are also supported by CFITSIO:
#define TINT 31 /* int */
#define TSBYTE 12 /* 8-bit signed byte, 'S' */
#define TUINT 30 /* unsigned int 'V' */
#define TUSHORT 20 /* unsigned short 'U' */
#define TULONG 40 /* unsigned long */
#define TLONGLONG 81 /* 64-bit long signed integer 'K' */
HDU type code values (value returned when moving to new HDU):
#define IMAGE_HDU 0 /* Primary Array or IMAGE HDU */
#define ASCII_TBL 1 /* ASCII table HDU */
#define BINARY_TBL 2 /* Binary table HDU */
#define ANY_HDU -1 /* matches any type of HDU */
Column name and string matching case-sensitivity:
#define CASESEN 1 /* do case-sensitive string match */
#define CASEINSEN 0 /* do case-insensitive string match */
Logical states (if TRUE and FALSE are not already defined):
#define TRUE 1
#define FALSE 0
Values to represent undefined floating point numbers:
#define FLOATNULLVALUE -9.11912E-36F
#define DOUBLENULLVALUE -9.1191291391491E-36
Image compression algorithm definitions
#define RICE_1 11
#define GZIP_1 21
#define PLIO_1 31
4.2 Current Header Data Unit (CHDU)
The concept of the Current Header and Data Unit, or CHDU, is
fundamental to the use of the CFITSIO library. A simple FITS image may
only contain a single Header and Data unit (HDU), but in general FITS
files can contain multiple Header Data Units (also known as
`extensions'), concatenated one after the other in the file. The user
can specify which HDU should be initially opened at run time by giving
the HDU name or number after the root file name. For example,
'myfile.fits[4]' opens the 5th HDU in the file (note that the numbering
starts with 0), and 'myfile.fits[EVENTS] opens the HDU with the name
'EVENTS' (as defined by the EXTNAME or HDUNAME keywords). If no HDU is
specified then CFITSIO opens the first HDU (the primary array) by
default. The CFITSIO routines which read and write data only operate
within the opened HDU, Other CFITSIO routines are provided to move to
and open any other existing HDU within the FITS file or to append or
insert new HDUs in the FITS file.
4.3 Function Names and Variable Datatypes
Most of the CFITSIO routines have both a short name as well as a
longer descriptive name. The short name is only 5 or 6 characters long
and is similar to the subroutine name in the Fortran-77 version of
FITSIO. The longer name is more descriptive and it is recommended that
it be used instead of the short name to more clearly document the
source code.
Many of the CFITSIO routines come in families which differ only in the
data type of the associated parameter(s). The data type of these
routines is indicated by the suffix of the routine name. The short
routine names have a 1 or 2 character suffix (e.g., 'j' in 'ffpkyj')
while the long routine names have a 4 character or longer suffix
as shown in the following table:
Long Short Data
Names Names Type
----- ----- ----
_bit x bit
_byt b unsigned byte
_sbyt sb signed byte
_sht i short integer
_lng j long integer
_lnglng jj 8-byte LONGLONG integer (see note below)
_usht ui unsigned short integer
_ulng uj unsigned long integer
_uint uk unsigned int integer
_int k int integer
_flt e real exponential floating point (float)
_fixflt f real fixed-decimal format floating point (float)
_dbl d double precision real floating-point (double)
_fixdbl g double precision fixed-format floating point (double)
_cmp c complex reals (pairs of float values)
_fixcmp fc complex reals, fixed-format floating point
_dblcmp m double precision complex (pairs of double values)
_fixdblcmp fm double precision complex, fixed-format floating point
_log l logical (int)
_str s character string
The logical data type corresponds to `int' for logical keyword values,
and `byte' for logical binary table columns. In other words, the value
when writing a logical keyword must be stored in an `int' variable, and
must be stored in a `char' array when reading or writing to `L' columns
in a binary table. Inplicit data type conversion is not supported for
logical table columns, but is for keywords, so a logical keyword may be
read and cast to any numerical data type; a returned value = 0
indicates false, and any other value = true.
The `int' data type may be 2 bytes long on some IBM PC compatible
systems and is usually 4 bytes long on most other systems. Some 64-bit
machines, however, like the Dec Alpha/OSF, define the `short', `int',
and `long' integer data types to be 2, 4, and 8 bytes long,
respectively. The FITS standard only supports 2 and 4 byte integer
data types, so CFITSIO internally converts between 4 and 8 bytes when
reading or writing `long' integers on Alpha/OSF systems.
The 8-byte 'LONGLONG' integer data type is supported on most platforms.
CFITSIO defines the LONGLONG data type to be equivalent to 'long long'
on most Unix platforms and on Mac OS-X. Since most Windows compilers don't
support the 'long long' data type, LONGLONG is defined instead to be
equivalent to '__int64'. If the compiler does not support a 8-byte
integer data type then LONGLONG is defined to be equivalent to 'long'.
Note that the C format specifier to print out these long integer values
is "%lld" on most unix machines, except on OSF platforms where "%ld"
must be used. On Windows platform which have the __int64 data type,
the format specifier is "%INT64d".
When dealing with the FITS byte data type it is important to remember
that the raw values (before any scaling by the BSCALE and BZERO, or
TSCALn and TZEROn keyword values) in byte arrays (BITPIX = 8) or byte
columns (TFORMn = 'B') are interpreted as unsigned bytes with values
ranging from 0 to 255. Some C compilers define a 'char' variable as
signed, so it is important to explicitly declare a numeric char
variable as 'unsigned char' to avoid any ambiguity
One feature of the CFITSIO routines is that they can operate on a `X'
(bit) column in a binary table as though it were a `B' (byte) column.
For example a `11X' data type column can be interpreted the same as a
`2B' column (i.e., 2 unsigned 8-bit bytes). In some instances, it can
be more efficient to read and write whole bytes at a time, rather than
reading or writing each individual bit.
The complex and double precision complex data types are not directly
supported in ANSI C so these data types should be interpreted as pairs
of float or double values, respectively, where the first value in each
pair is the real part, and the second is the imaginary part.
4.4 Support for Unsigned Integers and Signed Bytes
Although FITS does not directly support unsigned integers as one of its
fundamental data types, FITS can still be used to efficiently store
unsigned integer data values in images and binary tables. The
convention used in FITS files is to store the unsigned integers as
signed integers with an associated offset (specified by the BZERO or
TZEROn keyword). For example, to store unsigned 16-bit integer values
in a FITS image the image would be defined as a signed 16-bit integer
(with BITPIX keyword = SHORT_IMG = 16) with the keywords BSCALE = 1.0
and BZERO = 32768. Thus the unsigned values of 0, 32768, and 65535,
for example, are physically stored in the FITS image as -32768, 0, and
32767, respectively; CFITSIO automatically adds the BZERO offset to
these values when they are read. Similarly, in the case of unsigned
32-bit integers the BITPIX keyword would be equal to LONG_IMG = 32 and
BZERO would be equal to 2147483648 (i.e. 2 raised to the 31st power).
The CFITSIO interface routines will efficiently and transparently apply
the appropriate offset in these cases so in general application
programs do not need to be concerned with how the unsigned values are
actually stored in the FITS file. As a convenience for users, CFITSIO
has several predefined constants for the value of BITPIX (USHORT_IMG,
ULONG_IMG) and for the TFORMn value in the case of binary tables (`U'
and `V') which programmers can use when creating FITS files containing
unsigned integer values. The following code fragment illustrates how
to write a FITS 1-D primary array of unsigned 16-bit integers:
unsigned short uarray[100];
int naxis, status;
long naxes[10], group, firstelem, nelements;
...
status = 0;
naxis = 1;
naxes[0] = 100;
fits_create_img(fptr, USHORT_IMG, naxis, naxes, &status);
firstelem = 1;
nelements = 100;
fits_write_img(fptr, TUSHORT, firstelem, nelements,
uarray, &status);
...
In the above example, the 2nd parameter in fits_create_img tells
CFITSIO to write the header keywords appropriate for an array of 16-bit
unsigned integers (i.e., BITPIX = 16 and BZERO = 32768). Then the
fits_write_img routine writes the array of unsigned short integers
(uarray) into the primary array of the FITS file. Similarly, a 32-bit
unsigned integer image may be created by setting the second parameter
in fits_create_img equal to `ULONG_IMG' and by calling the
fits_write_img routine with the second parameter = TULONG to write
the array of unsigned long image pixel values.
An analogous set of routines are available for reading or writing unsigned
integer values and signed byte values in a FITS binary table extension.
When specifying the TFORMn keyword value which defines the format of a
column, CFITSIO recognized 3 additional data type codes besides those
already defined in the FITS standard: `U' meaning a 16-bit unsigned
integer column, `V' for a 32-bit unsigned integer column, and 'S'
for a signed byte column. These non-standard data type codes are not
actually written into the FITS file but instead are just used internally
within CFITSIO. The following code fragment illustrates how to use
these features:
unsigned short uarray[100];
unsigned int varray[100];
int colnum, tfields, status;
long nrows, firstrow, firstelem, nelements, pcount;
char extname[] = "Test_table"; /* extension name */
/* define the name, data type, and physical units for the 2 columns */
char *ttype[] = { "Col_1", "Col_2", "Col_3" };
char *tform[] = { "1U", "1V", "1S"}; /* special CFITSIO codes */
char *tunit[] = { " ", " ", " " };
...
/* write the header keywords */
status = 0;
nrows = 1;
tfields = 3
pcount = 0;
fits_create_tbl(fptr, BINARY_TBL, nrows, tfields, ttype, tform,
tunit, extname, &status);
/* write the unsigned shorts to the 1st column */
colnum = 1;
firstrow = 1;
firstelem = 1;
nelements = 100;
fits_write_col(fptr, TUSHORT, colnum, firstrow, firstelem,
nelements, uarray, &status);
/* now write the unsigned longs to the 2nd column */
colnum = 2;
fits_write_col(fptr, TUINT, colnum, firstrow, firstelem,
nelements, varray, &status);
...
Note that the non-standard TFORM values for the 3 columns, `U' and `V',
tell CFITSIO to write the keywords appropriate for unsigned 16-bit and
unsigned 32-bit integers, respectively (i.e., TFORMn = '1I' and TZEROn
= 32678 for unsigned 16-bit integers, and TFORMn = '1J' and TZEROn =
2147483648 for unsigned 32-bit integers). The 'S' TFORMn value tells
CFITSIO to write the keywords appropriate for a signed 8-bit byte column
with TFORMn = '1B' and TZEROn = -128. The calls to fits_write_col
then write the arrays of unsigned integer values to the columns.
4.5 Dealing with Character Strings
The character string values in a FITS header or in an ASCII column in a
FITS table extension are generally padded out with non-significant
space characters (ASCII 32) to fill up the header record or the column
width. When reading a FITS string value, the CFITSIO routines will
strip off these non-significant trailing spaces and will return a
null-terminated string value containing only the significant
characters. Leading spaces in a FITS string are considered
significant. If the string contains all blanks, then CFITSIO will
return a single blank character, i.e, the first blank is considered to
be significant, since it distinguishes the string from a null or
undefined string, but the remaining trailing spaces are not
significant.
Similarly, when writing string values to a FITS file the
CFITSIO routines expect to get a null-terminated string as input;
CFITSIO will pad the string with blanks if necessary when writing it
to the FITS file.
When calling CFITSIO routines that return a character string it is
vital that the size of the char array be large enough to hold the
entire string of characters, otherwise CFITSIO will overwrite whatever
memory locations follow the char array, possibly causing the program to
execute incorrectly. This type of error can be difficult to debug, so
programmers should always ensure that the char arrays are allocated
enough space to hold the longest possible string, including the
terminating NULL character. The fitsio.h file contains the following
defined constants which programmers are strongly encouraged to use
whenever they are allocating space for char arrays:
#define FLEN_FILENAME 1025 /* max length of a filename */
#define FLEN_KEYWORD 72 /* max length of a keyword */
#define FLEN_CARD 81 /* length of a FITS header card */
#define FLEN_VALUE 71 /* max length of a keyword value string */
#define FLEN_COMMENT 73 /* max length of a keyword comment string */
#define FLEN_ERRMSG 81 /* max length of a CFITSIO error message */
#define FLEN_STATUS 31 /* max length of a CFITSIO status text string */
For example, when declaring a char array to hold the value string
of FITS keyword, use the following statement:
char value[FLEN_VALUE];
Note that FLEN_KEYWORD is longer than needed for the nominal 8-character
keyword name because the HIERARCH convention supports longer keyword names.
4.6 Implicit Data Type Conversion
The CFITSIO routines that read and write numerical data can perform
implicit data type conversion. This means that the data type of the
variable or array in the program does not need to be the same as the
data type of the value in the FITS file. Data type conversion is
supported for numerical and string data types (if the string contains a
valid number enclosed in quotes) when reading a FITS header keyword
value and for numeric values when reading or writing values in the
primary array or a table column. CFITSIO returns status =
NUM_OVERFLOW if the converted data value exceeds the range of the
output data type. Implicit data type conversion is not supported
within binary tables for string, logical, complex, or double complex
data types.
In addition, any table column may be read as if it contained string values.
In the case of numeric columns the returned string will be formatted
using the TDISPn display format if it exists.
4.7 Data Scaling
When reading numerical data values in the primary array or a
table column, the values will be scaled automatically by the BSCALE and
BZERO (or TSCALn and TZEROn) header values if they are
present in the header. The scaled data that is returned to the reading
program will have
output value = (FITS value) * BSCALE + BZERO
(a corresponding formula using TSCALn and TZEROn is used when reading
from table columns). In the case of integer output values the floating
point scaled value is truncated to an integer (not rounded to the
nearest integer). The fits_set_bscale and fits_set_tscale routines
(described in the `Advanced' chapter) may be used to override the
scaling parameters defined in the header (e.g., to turn off the scaling
so that the program can read the raw unscaled values from the FITS
file).
When writing numerical data to the primary array or to a table column
the data values will generally be automatically inversely scaled by the
value of the BSCALE and BZERO (or TSCALn and TZEROn) keyword values if
they they exist in the header. These keywords must have been written
to the header before any data is written for them to have any immediate
effect. One may also use the fits_set_bscale and fits_set_tscale
routines to define or override the scaling keywords in the header
(e.g., to turn off the scaling so that the program can write the raw
unscaled values into the FITS file). If scaling is performed, the
inverse scaled output value that is written into the FITS file will
have
FITS value = ((input value) - BZERO) / BSCALE
(a corresponding formula using TSCALn and TZEROn is used when
writing to table columns). Rounding to the nearest integer, rather
than truncation, is performed when writing integer data types to the
FITS file.
4.8 Support for IEEE Special Values
The ANSI/IEEE-754 floating-point number standard defines certain
special values that are used to represent such quantities as
Not-a-Number (NaN), denormalized, underflow, overflow, and infinity.
(See the Appendix in the NOST FITS standard or the NOST FITS User's
Guide for a list of these values). The CFITSIO routines that read
floating point data in FITS files recognize these IEEE special values
and by default interpret the overflow and infinity values as being
equivalent to a NaN, and convert the underflow and denormalized values
into zeros. In some cases programmers may want access to the raw IEEE
values, without any modification by CFITSIO. This can be done by
calling the fits_read_img or fits_read_col routines while
specifying 0.0 as the value of the NULLVAL parameter. This will force
CFITSIO to simply pass the IEEE values through to the application
program without any modification. This is not fully supported on
VAX/VMS machines, however, where there is no easy way to bypass the
default interpretation of the IEEE special values.
4.9 Error Status Values and the Error Message Stack
Nearly all the CFITSIO routines return an error status value
in 2 ways: as the value of the last parameter in the function call,
and as the returned value of the function itself. This provides
some flexibility in the way programmers can test if an error
occurred, as illustrated in the following 2 code fragments:
if ( fits_write_record(fptr, card, &status) )
printf(" Error occurred while writing keyword.");
or,
fits_write_record(fptr, card, &status);
if ( status )
printf(" Error occurred while writing keyword.");
A listing of all the CFITSIO status code values is given at the end of
this document. Programmers are encouraged to use the symbolic
mnemonics (defined in fitsio.h) rather than the actual integer status
values to improve the readability of their code.
The CFITSIO library uses an `inherited status' convention for the
status parameter which means that if a routine is called with a
positive input value of the status parameter as input, then the routine
will exit immediately without changing the value of the status
parameter. Thus, if one passes the status value returned from each
CFITSIO routine as input to the next CFITSIO routine, then whenever an
error is detected all further CFITSIO processing will cease. This
convention can simplify the error checking in application programs
because it is not necessary to check the value of the status parameter
after every single CFITSIO routine call. If a program contains a
sequence of several CFITSIO calls, one can just check the status value
after the last call. Since the returned status values are generally
distinctive, it should be possible to determine which routine
originally returned the error status.
CFITSIO also maintains an internal stack of error messages
(80-character maximum length) which in many cases provide a more
detailed explanation of the cause of the error than is provided by the
error status number alone. It is recommended that the error message
stack be printed out whenever a program detects a CFITSIO error. The
function fits_report_error will print out the entire error message
stack, or alternatively one may call fits_read_errmsg to get the
error messages one at a time.
4.10 Variable-Length Arrays in Binary Tables
CFITSIO provides easy-to-use support for reading and writing data in
variable length fields of a binary table. The variable length columns
have TFORMn keyword values of the form `1Pt(len)' where `t' is the
data type code (e.g., I, J, E, D, etc.) and `len' is an integer
specifying the maximum length of the vector in the table. If the value
of `len' is not specified when the table is created (e.g., if the TFORM
keyword value is simply specified as '1PE' instead of '1PE(400) ), then
CFITSIO will automatically scan the table when it is closed to
determine the maximum length of the vector and will append this value
to the TFORMn value.
The same routines which read and write data in an ordinary fixed length
binary table extension are also used for variable length fields,
however, the routine parameters take on a slightly different
interpretation as described below.
All the data in a variable length field is written into an area called
the `heap' which follows the main fixed-length FITS binary table. The
size of the heap, in bytes, is specified by the PCOUNT keyword in the
FITS header. When creating a new binary table, the initial value of
PCOUNT should usually be set to zero. CFITSIO will recompute the size
of the heap as the data is written and will automatically update the
PCOUNT keyword value when the table is closed. When writing variable
length data to a table, CFITSIO will automatically extend the size
of the heap area if necessary, so that any following HDUs do not
get overwritten.
By default the heap data area starts immediately after the last row of
the fixed-length table. This default starting location may be
overridden by the THEAP keyword, but this is not recommended.
If additional rows of data are added to the table, CFITSIO will
automatically shift the the heap down to make room for the new
rows, but it is obviously be more efficient to initially
create the table with the necessary number of blank rows, so that
the heap does not needed to be constantly moved.
When writing to a variable length field the entire array of values for
a given row of the table must be written with a single call to
fits_write_col. The total length of the array is given by nelements
+ firstelem - 1. Additional elements cannot be appended to an existing
vector at a later time since any attempt to do so will simply overwrite
all the previously written data. Note also that the new data will be
written to a new area of the heap and the heap space used by the
previous write cannot be reclaimed. For this reason each row of a
variable length field should only be written once. An exception to
this general rule occurs when setting elements of an array as
undefined. One must first write a dummy value into the array with
fits_write_col, and then call fits_write_col_nul to flag the
desired elements as undefined. (Do not use the fits_write_colnull
routines with variable length fields). Note that the rows of a table,
whether fixed or variable length, do not have to be written
consecutively and may be written in any order.
When writing to a variable length ASCII character field (e.g., TFORM =
'1PA') only a single character string can be written. The `firstelem'
and `nelements' parameter values in the fits_write_col routine are
ignored and the number of characters to write is simply determined by
the length of the input null-terminated character string.
The fits_write_descript routine is useful in situations where
multiple rows of a variable length column have the identical array of
values. One can simply write the array once for the first row, and
then use fits_write_descript to write the same descriptor values into
the other rows; all the rows will then point to the same storage
location thus saving disk space.
When reading from a variable length array field one can only read as
many elements as actually exist in that row of the table; reading does
not automatically continue with the next row of the table as occurs
when reading an ordinary fixed length table field. Attempts to read
more than this will cause an error status to be returned. One can
determine the number of elements in each row of a variable column with
the fits_read_descript routine.
4.11 Multiple Access to the Same FITS File
CFITSIO supports simultaneous read and write access to multiple HDUs in
the same FITS file. Thus, one can open the same FITS file twice within
a single program and move to 2 different HDUs in the file, and then
read and write data or keywords to the 2 extensions just as if one were
accessing 2 completely separate FITS files. Since in general it is
not possible to physically open the same file twice and then expect to
be able to simultaneously (or in alternating succession) write to 2
different locations in the file, CFITSIO recognizes when the file to be
opened (in the call to fits_open_file) has already been opened and
instead of actually opening the file again, just logically links the
new file to the old file. (This only applies if the file is opened
more than once within the same program, and does not prevent the same
file from being simultaneously opened by more than one program). Then
before CFITSIO reads or writes to either (logical) file, it makes sure
that any modifications made to the other file have been completely
flushed from the internal buffers to the file. Thus, in principle, one
could open a file twice, in one case pointing to the first extension
and in the other pointing to the 2nd extension and then write data to
both extensions, in any order, without danger of corrupting the file,
There may be some efficiency penalties in doing this however, since
CFITSIO has to flush all the internal buffers related to one file
before switching to the other, so it would still be prudent to
minimize the number of times one switches back and forth between doing
I/O to different HDUs in the same file.
4.12 When the Final Size of the FITS HDU is Unknown
It is not required to know the total size of a FITS data array or table
before beginning to write the data to the FITS file. In the case of
the primary array or an image extension, one should initially create
the array with the size of the highest dimension (largest NAXISn
keyword) set to a dummy value, such as 1. Then after all the data have
been written and the true dimensions are known, then the NAXISn value
should be updated using the fits_update_key routine before moving to
another extension or closing the FITS file.
When writing to FITS tables, CFITSIO automatically keeps track of the
highest row number that is written to, and will increase the size of
the table if necessary. CFITSIO will also automatically insert space
in the FITS file if necessary, to ensure that the data 'heap', if it
exists, and/or any additional HDUs that follow the table do not get
overwritten as new rows are written to the table.
As a general rule it is best to specify the initial number of rows = 0
when the table is created, then let CFITSIO keep track of the number of
rows that are actually written. The application program should not
manually update the number of rows in the table (as given by the NAXIS2
keyword) since CFITSIO does this automatically. If a table is
initially created with more than zero rows, then this will usually be
considered as the minimum size of the table, even if fewer rows are
actually written to the table. Thus, if a table is initially created
with NAXIS2 = 20, and CFITSIO only writes 10 rows of data before
closing the table, then NAXIS2 will remain equal to 20. If however, 30
rows of data are written to this table, then NAXIS2 will be increased
from 20 to 30. The one exception to this automatic updating of the
NAXIS2 keyword is if the application program directly modifies the
value of NAXIS2 (up or down) itself just before closing the table. In this
case, CFITSIO does not update NAXIS2 again, since it assumes that the
application program must have had a good reason for changing the value
directly. This is not recommended, however, and is only provided for
backward compatibility with software that initially creates a table
with a large number of rows, than decreases the NAXIS2 value to the
actual smaller value just before closing the table.
4.13 CFITSIO Size Limitations
CFITSIO places very few restrictions on the size of FITS files that it
reads or writes. There are a few limits, however, which may affect
some extreme cases:
1. The maximum number of FITS files that may be simultaneously opened
by CFITSIO is set by NMAXFILES as defined in fitsio2.h. It is currently
set = 300 by default. CFITSIO will allocate about 80 * NMAXFILES bytes
of memory for internal use. Note that the underlying C compiler or
operating system, may have a smaller limit on the number of opened files.
The C symbolic constant FOPEN_MAX is intended to define the maximum
number of files that may open at once (including any other text or
binary files that may be open, not just FITS files). On some systems it
has been found that gcc supports a maximum of 255 opened files.
Note that opening and operating on many FITS files simultaneously in
parallel may be less efficient than operating on smaller groups of files
in series. CFITSIO only has NIOBUF number of internal buffers (set = 40
by default) that are used for temporary storage of the most recent data
records that have been read or written in the FITS files. If the number
of opened files is greater than NIOBUF, then CFITSIO may waste more time
flushing and re-reading or re-writing the same records in the FITS files.
2. By default, CFITSIO can handle FITS files up to 2.1 GB in size
(2**31 bytes). This file size limit is often imposed by 32-bit
operating systems. More recently, as 64-bit operating systems become
more common, an industry-wide standard (at least on Unix systems) has
been developed to support larger sized files (see
http://ftp.sas.com/standards/large.file/). Starting with version 2.1
of CFITSIO, larger FITS files up to 6 terabytes in size may be read and
written on certain supported platforms. In order to support these
larger files, CFITSIO must be compiled with the
`-D_FILE_OFFSET_BITS=64' compiler flag. All programs which link to
the CFITSIO library must also be compiled with this flag or must
include this preprocessor definition at the start of the source code
file. This causes the compiler to allocate 8-bytes instead of 4-bytes
for the `off_t' data type which is used to store file offset
positions.
If CFITSIO is compiled with the -D_FILE_OFFSET_BITS=64 flag on a
platform that supports large files, then it can read and write FITS
files that contain up to 2**31 2880-byte FITS records, or approximately
6 terabytes in size. It is still required that the value of the NAXISn
and PCOUNT keywords in each extension be within the range of a signed
4-byte integer (max value = 2,147,483,648). Thus, each dimension of an
image (given by the NAXISn keywords), the total width of a table
(NAXIS1 keyword), the number of rows in a table (NAXIS2 keyword), and
the total size of the variable-length array heap in binary tables
(PCOUNT keyword) must be less than this limit.
Currently, support for large files within CFITSIO has only been tested
on the Solaris 2.6 operating system using the Sun cc compiler or gcc
2.95.2.