Rationale

Contents

Design Goals
Design Decisions
Generic Design
Modes
Chain Interface
Return Type of write
Stream Positioning
Planned Changes
Future Directions

Design Goals

The Boost Iostreams library was designed with the following criteria in mind:
  1. The library should be easy to learn, use and extend.
  2. The library should be flexible enough to produce streams and stream buffers with almost any desired characteristics.
  3. Streams and stream buffers produced using the library should be as efficient as hand-written components, in most cases.[1]
  4. The library should fit easily within the framework of the C++ standard library, in the following two ways:
    1. Streams and stream buffers constructed using the library should not, to the extent possible, require a richer interface than standard library streams and stream buffers in order to expose their full functionality.
    2. The components used to construct streams and stream buffers — Source, Sinks, InputFilters and OutputFilters — should not, to the extent possible, requirer a richer interface than standard library streams and stream buffers.

Roughly, item 4a. says that streams and stream buffers should be usable wherever standard library streams and stream buffers are usable; item 4b. says that standard library streams and stream buffers should be usable wherever Source and Sinks are usable.

Design Decisions

Generic Design

The benefits of generic design are well-known. As it relates to the current library, the main benefits are these:

  1. Existing components — such as streams, stream buffers and iterators — can be integrated easily into the library.
  2. The library can be easily extended to handle new Filter and Device concepts with additional functionality.
  3. The internals of the library, such as the implementation of Filter chains as lists of standard stream buffers, can be redesigned without requiring user code to change.

One benefit of generic design which is often cited but which has minimal impact on the current library is the performance boost that comes when virtual function calls are replaced by inline code. Because std::basic_streambuf is implemented in terms of virtual functions we cannot resonably expect to eliminate virtual function calls. The cost of these calls, however, is largely mitigated by buffering. In fact, because of buffering, we might reduce code bloat by internally wrapping Filters and Devices in polymorphic classes with virtual function read, write, etc. without a noticable performance impact.

Modes

The large number of supported modes probably adds more than anything else to the complexity of the library. Unfortunately, all the modes seem to be necessary for a comprehesive iostreams extension framework. The examples here indicate that all but one of the modes have important use cases. The exception is bidirectional_seekable; this mode, however, essentially comes for free once the others are implemented.

I/O libraries tend to handle this situation in one of two ways:

  1. Only allow for input and output.
  2. Allow for components with other modes but don't provide direct library support. Users must consult the documentation for individual components to determine whether they are compatible.

Clearly the second alternative is unacceptable. The first alternative is attractive for its simplicity, but leaves out crucial use cases. It was judged that the library could achieve the best of both worlds simply by recommending that new users ignore all modes but input and output.

Chain Interface

In earlier versions of this library, the chain interface was much richer. Chains had iterators and could be manipulated much like std::lists. The class detail::chain was part of the public interface of the library so that one could prepare a chain of Filters and use it with several streams in succession.

This functionality was removed simply to make the library easier to learn. If there is a need for it, it can easily be restored.

Return Type of write

Originally, the function boost::iostreams::write was specified to return a std::streamsize value indicating how many characters were successfully written. The current specification, that write should throw an exception if the requested number of characters cannot be written, was adopter for these reasons:

  1. Writing fewer than the requested number of characters almost always represents an error;
  2. There is typically not much that can be done to address the error locally; and
  3. Making Filter and Device authors repeat the std::streamsize function argument in a return statement is tedious and error-prone.

This is being reconsidered in the context of alternative models. See Planned Changes and Future Directions.

Stream Positioning

For simplicity the library does not support stream positioning which is sensitive to code conversion state or streams and stream buffers which are repositional but not arbitrarily repositional in the sense of [ISO] 17.1.1 and 17.1.16. It should be straightforward to add this functionality if it is desired.

Planned Changes

The following changes are planned, but are on hold pending review:

  1. The mode dual_seekable will be made to refine seekable
  2. On conformant compilers, the need for the function adapt will be eliminated as follows. The concept of a smart Devices, which already exists internally, will be made public. Smart Devices adjust their character type and mode as they are added to a stream or stream buffer. This allows, e.g., output iterators to be treated as sinks even though their character type is not known in advance. Iterator ranges from Boost.Range, output iterators (identified using boost::detail::is_incrementable) and standard streams and stream buffers will then be recognized as smart Devices, with no need for explicit adaptation.
  3. Filters will be given some means to indicate that the number of characters written or read was fewer than requested, although end-of-sequence has not been reached and no error has occurred. This is necessary to accommodate models just as non-blocking, asynchronous and multiplexed.

Future Directions

There are a number of small additions to the library which might be beneficial:

  1. Give Filters and Devices a way to specify an optimal buffer size. The implementation is trivial; the principle difficulty is finding good names.
  2. Add generic functions for binary i/o. E.g.,
        template<typename Sink>
        void write_int32(Sink& sink, int);
  3. Change the definition of Direct Devices so that in addition to allowing for efficient direct access to in-memory arrays they can also be used with the functions read, write, putback, etc.

A more ambitious extension would be to add new Device concepts for non-blocking, asynchronous and multiplexed i/o, and provide appropriate abstractions for accessing these Devices, directly and with Filter chains. In order to make this work, it is necessary to ensure that the current Filter concepts can support these alternative models, since it would be wasteful to require several versions of each Filter to accommodate the various models. The main impediment is that Filters must be able to indicate that the number of characters written or read was fewer than requested, although end-of-sequence has not been reached and no error has occurred. There are several ways this could be achieved:

  1. Adopt the convention that reads and writes always block until at least one character is written. Let read return 0 to indicate end-of-sequence and let all other return values for read and write indicate the actual number of characters processed. get and put will always block.
  2. Let read return -1 to indicate end-of-sequence, and let all other return values, including 0, indicate the actual number of characters read or written. Let put return a bool indicating whether a character was written. Let the return type of get be a class type which can store a character, an end-of-sequence indicator or a 'try back later' indicator. E.g.,
        template<typename Ch>
        struct basic_character {
            enum nochar { eof, unavail };
    
            basic_character(Ch c);
            basic_character(nochar type);
            operator Ch () const;
    
            // Tests whether value represents a character.
            operator safe_bool() const;
            bool operator!() const;
    
            bool eof() const;
            bool unavail() const;
    
            ...
        };
        typedef basic_character<char>    character;
        typedef basic_character<wchar_t> wcharacter;
    
        character eof() { return character(character::eof); }
        wcharacter weof() { return wcharacter(character::eof); }
    
        character unavail() { return character(character::unavail); }
        wcharacter wunavail() { return wcharacter(character::unavail); }
    Using basic_character, writing InputFilters would be almost the same as it is currently.
  3. Designate an exception type to indicate 'try back later'.

None of these is entirely satisfactory. Item 1 allows performace to suffer — especially if Filters are involved — just to accommodate an insufficiently expressive interface. Item 2 makes writing Filters for ordinary blocking i/o slightly more complex, and may have an abstraction penalty. Item 3 is obviously unworkable.


[1]Informal tests conducted by the author showed stream<file_descriptor_sink> to be slightly faster than the Dinkumware implementation of std::ofstream which ships with Microsoft Visual Studio .NET 2003. Note, however, that a file_descriptor_sink performs no locking. Similarly, an ostringstream constructed with the Boost Iostreams library performed comparably to the Dinkumware implementation of std::stringstream. No tests of input or random access have been performed yet.