All streams of data in GStreamer are chopped up into chunks that are passed from a source pad on one element to a sink pad on another element. Data are structures used to hold these chunks of data.
Data contains the following important types:
An exact type indicating what type of data (control, content, ...) this Data is.
A reference count indicating the number of elements currently holding a reference to the buffer. When the buffer reference count falls to zero, the buffer will be unlinked, and its memory will be freed in some sense (see below for more details).
There are two types of data defined: events (control) and buffers (content).
Buffers may contain any sort of data that the two linked pads know how to handle. Normally, a buffer contains a chunk of some sort of audio or video data that flows from one element to another.
Buffers also contain metadata describing the buffer's contents. Some of the important types of metadata are:
A pointer to the buffer's data.
An integer indicating the size of the buffer's data.
A timestamp indicating the preferred display timestamp of the content in the buffer.
Events contain information on the state of the stream flowing between the two linked pads. Events will only be sent if the element explicitely supports them, else the core will (try to) handle the events automatically. Events are used to indicate, for example, a clock discontinuity, the end of a media stream or that the cache should be flushed.
Events may contain several of the following items:
A subtype indicating the type of the contained event.
The other contents of the event depend on the specific event type.
Events will be discussed extensively in Chapter 19. Until then, the only event that will be used is the EOS event, which is used to indicate the end-of-stream (usually end-of-file).
See the GStreamer Library Reference for the current implementation details of a GstMiniObject
, GstBuffer
and GstEvent
.
Buffers are able to store chunks of memory of several different types. The most generic type of buffer contains memory allocated by malloc(). Such buffers, although convenient, are not always very fast, since data often needs to be specifically copied into the buffer.
Many specialized elements create buffers that point to special memory. For example, the filesrc element usually maps a file into the address space of the application (using mmap()), and creates buffers that point into that address range. These buffers created by filesrc act exactly like generic buffers, except that they are read-only. The buffer freeing code automatically determines the correct method of freeing the underlying memory. Downstream elements that recieve these kinds of buffers do not need to do anything special to handle or unreference it.
Another way an element might get specialized buffers is to request them from a downstream peer. These are called downstream-allocated buffers. Elements can ask a peer connected to a source pad to create an empty buffer of a given size. If a downstream element is able to create a special buffer of the correct size, it will do so. Otherwise GStreamer will automatically create a generic buffer instead. The element that requested the buffer can then copy data into the buffer, and push the buffer to the source pad it was allocated from.
Many sink elements have accelerated methods for copying data to hardware, or have direct access to hardware. It is common for these elements to be able to create downstream-allocated buffers for their upstream peers. One such example is ximagesink. It creates buffers that contain XImages. Thus, when an upstream peer copies data into the buffer, it is copying directly into the XImage, enabling ximagesink to draw the image directly to the screen instead of having to copy data into an XImage first.
Filter elements often have the opportunity to either work on a buffer in-place, or work while copying from a source buffer to a destination buffer. It is optimal to implement both algorithms, since the GStreamer framework can choose the fastest algorithm as appropriate. Naturally, this only makes sense for strict filters -- elements that have exactly the same format on source and sink pads.