Other Kinds of Streams

The GNU library provides ways for you to define additional kinds of streams that do not necessarily correspond to an open file.

One such type of stream takes input from or writes output to a string. These kinds of streams are used internally to implement the sprintf and sscanf functions. You can also create such a stream explicitly, using the functions described in the section called “String Streams”.

More generally, you can define streams that do input/output to arbitrary objects using functions supplied by your program. This protocol is discussed in the section called “Programming Your Own Custom Streams”.

Portability Note: The facilities described in this section are specific to GNU. Other systems or C implementations might or might not provide equivalent functionality.

String Streams

The fmemopen and open_memstream functions allow you to do I/O to a string or memory buffer. These facilities are declared in stdio.h. FILE * function>fmemopen/function> (void *buf, size_t size, const char *opentype) This function opens a stream that allows the access specified by the opentype argument, that reads from or writes to the buffer specified by the argument buf. This array must be at least size bytes long.

If you specify a null pointer as the buf argument, fmemopen dynamically allocates an array size bytes long (as with malloc; the section called “Unconstrained Allocation”). This is really only useful if you are going to write things to the buffer and then read them back in again, because you have no way of actually getting a pointer to the buffer (for this, try open_memstream, below). The buffer is freed when the stream is closed.

The argument opentype is the same as in fopen (the section called “Opening Streams”). If the opentype specifies append mode, then the initial file position is set to the first null character in the buffer. Otherwise the initial file position is at the beginning of the buffer.

When a stream open for writing is flushed or closed, a null character (zero byte) is written at the end of the buffer if it fits. You should add an extra byte to the size argument to account for this. Attempts to write more than size bytes to the buffer result in an error.

For a stream open for reading, null characters (zero bytes) in the buffer do not count as "end of file". Read operations indicate end of file only when the file position advances past size bytes. So, if you want to read characters from a null-terminated string, you should supply the length of the string as the size argument.

Here is an example of using fmemopen to create a stream for reading from a string:

#include stdio.h

static char buffer[] = "foobar";

int
main (void)
{
  int ch;
  FILE *stream;

  stream = fmemopen (buffer, strlen (buffer), "r");
  while ((ch = fgetc (stream)) != EOF)
    printf ("Got %c\n", ch);
  fclose (stream);

  return 0;
}

This program produces the following output:

Got f
Got o
Got o
Got b
Got a
Got r

FILE * function>open_memstream/function> (char **ptr, size_t *sizeloc) This function opens a stream for writing to a buffer. The buffer is allocated dynamically (as with malloc; the section called “Unconstrained Allocation”) and grown as necessary.

When the stream is closed with fclose or flushed with fflush, the locations ptr and sizeloc are updated to contain the pointer to the buffer and its size. The values thus stored remain valid only as long as no further output on the stream takes place. If you do more output, you must flush the stream again to store new values before you use them again.

A null character is written at the end of the buffer. This null character is not included in the size value stored at sizeloc.

You can move the stream's file position with fseek or fseeko (the section called “File Positioning”). Moving the file position past the end of the data already written fills the intervening space with zeroes.

Here is an example of using open_memstream:

#include stdio.h

int
main (void)
{
  char *bp;
  size_t size;
  FILE *stream;

  stream = open_memstream (bp, size);
  fprintf (stream, "hello");
  fflush (stream);
  printf ("buf = `%s', size = %d\n", bp, size);
  fprintf (stream, ", world");
  fclose (stream);
  printf ("buf = `%s', size = %d\n", bp, size);

  return 0;
}

This program produces the following output:

buf = `hello', size = 5
buf = `hello, world', size = 12

Obstack Streams

You can open an output stream that puts it data in an obstack. the section called “Obstacks”.

FILE * function>open_obstack_stream/function> (struct obstack *obstack) This function opens a stream for writing data into the obstack obstack. This starts an object in the obstack and makes it grow as data is written (the section called “Growing Objects ”).

Calling fflush on this stream updates the current size of the object to match the amount of data that has been written. After a call to fflush, you can examine the object temporarily.

You can move the file position of an obstack stream with fseek or fseeko (the section called “File Positioning”). Moving the file position past the end of the data written fills the intervening space with zeros.

To make the object permanent, update the obstack with fflush, and then use obstack_finish to finalize the object and get its address. The following write to the stream starts a new object in the obstack, and later writes add to that object until you do another fflush and obstack_finish.

But how do you find out how long the object is? You can get the length in bytes by calling obstack_object_size (the section called “Status of an Obstack”), or you can null-terminate the object like this:

obstack_1grow (obstack, 0);

Whichever one you do, you must do it before calling obstack_finish. (You can do both if you wish.)

Here is a sample function that uses open_obstack_stream:

char *
make_message_string (const char *a, int b)
{
  FILE *stream = open_obstack_stream (message_obstack);
  output_task (stream);
  fprintf (stream, ": ");
  fprintf (stream, a, b);
  fprintf (stream, "\n");
  fclose (stream);
  obstack_1grow (message_obstack, 0);
  return obstack_finish (message_obstack);
}

Programming Your Own Custom Streams

This section describes how you can make a stream that gets input from an arbitrary data source or writes output to an arbitrary data sink programmed by you. We call these custom streams. The functions and types described here are all GNU extensions.

Custom Streams and Cookies

Inside every custom stream is a special object called the cookie. This is an object supplied by you which records where to fetch or store the data read or written. It is up to you to define a data type to use for the cookie. The stream functions in the library never refer directly to its contents, and they don't even know what the type is; they record its address with type void *.

To implement a custom stream, you must specify how to fetch or store the data in the specified place. You do this by defining hook functions to read, write, change "file position", and close the stream. All four of these functions will be passed the stream's cookie so they can tell where to fetch or store the data. The library functions don't know what's inside the cookie, but your functions will know.

When you create a custom stream, you must specify the cookie pointer, and also the four hook functions stored in a structure of type cookie_io_functions_t.

These facilities are declared in stdio.h. function>cookie_io_functions_t/function> This is a structure type that holds the functions that define the communications protocol between the stream and its cookie. It has the following members:

cookie_read_function_t *read

This is the function that reads data from the cookie. If the value is a null pointer instead of a function, then read operations on this stream always return EOF.

cookie_write_function_t *write

This is the function that writes data to the cookie. If the value is a null pointer instead of a function, then data written to the stream is discarded.

cookie_seek_function_t *seek

This is the function that performs the equivalent of file positioning on the cookie. If the value is a null pointer instead of a function, calls to fseek or fseeko on this stream can only seek to locations within the buffer; any attempt to seek outside the buffer will return an ESPIPE error.

cookie_close_function_t *close

This function performs any appropriate cleanup on the cookie when closing the stream. If the value is a null pointer instead of a function, nothing special is done to close the cookie when the stream is closed.

FILE * function>fopencookie/function> (void *cookie, const char *opentype, cookie_io_functions_t io-functions) This function actually creates the stream for communicating with the cookie using the functions in the io-functions argument. The opentype argument is interpreted as for fopen; see the section called “Opening Streams”. (But note that the "truncate on open" option is ignored.) The new stream is fully buffered.

The fopencookie function returns the newly created stream, or a null pointer in case of an error.

Custom Stream Hook Functions

Here are more details on how you should define the four hook functions that a custom stream needs.

You should define the function to read data from the cookie as:

ssize_t reader (void *cookie, char *buffer, size_t size)

This is very similar to the read function; see the section called “Input and Output Primitives”. Your function should transfer up to size bytes into the buffer, and return the number of bytes read, or zero to indicate end-of-file. You can return a value of -1 to indicate an error.

You should define the function to write data to the cookie as:

ssize_t writer (void *cookie, const char *buffer, size_t size)

This is very similar to the write function; see the section called “Input and Output Primitives”. Your function should transfer up to size bytes from the buffer, and return the number of bytes written. You can return a value of -1 to indicate an error.

You should define the function to perform seek operations on the cookie as:

int seeker (void *cookie, fpos_t *position, int whence)

For this function, the position and whence arguments are interpreted as for fgetpos; see the section called “Portable File-Position Functions”. In the GNU library, fpos_t is equivalent to off_t or long int, and simply represents the number of bytes from the beginning of the file.

After doing the seek operation, your function should store the resulting file position relative to the beginning of the file in position. Your function should return a value of 0 on success and -1 to indicate an error.

You should define the function to do cleanup operations on the cookie appropriate for closing the stream as:

int cleaner (void *cookie)

Your function should return -1 to indicate an error, and 0 otherwise.

function>cookie_read_function/function> This is the data type that the read function for a custom stream should have. If you declare the function as shown above, this is the type it will have.

function>cookie_write_function/function> The data type of the write function for a custom stream.

function>cookie_seek_function/function> The data type of the seek function for a custom stream.

function>cookie_close_function/function> The data type of the close function for a custom stream.