Raptor RDF Parser Toolkit - Release Notes

Raptor 1.4.7 Changes

Fix a couple of crashes in the RSS tag soup parser / serializer (Dave Beckett, Suzan Foster).

configure now looks for the libxslt/xslt.h header as well as the libxslt library and disables XSLT and GRDDL support it if is missing. This catches systems with the libraries without headers as has happened on some OSX versions.

In serializers rdfxml and rdfxml-abbrev, report failure to serialize to RDF/XML if the predicate URI is not absolute.

Raptor 1.4.6 Changes

Added --with-xslt-config configure option

Added a new parser for Gleaning Resource Descriptions from Dialects of Languages (GRDDL) which allows reading XHTML and XML as RDF triples by using profiles in the document that declare XSLT transforms from the XHTML/XML content into RDF/XML which is the RDF content. It does not support all the GRDDL styles, for example dataview:namespaceTransformation, or perform recursive transformations.

The turtle parser now accepts """long literals"""

XML writer feature support were added in 1.4.5 and not documented. The new functions are: raptor_xml_writer_features_enumerate, raptor_xml_writer_set_feature, raptor_xml_writer_set_feature_string, raptor_xml_writer_get_feature and raptor_xml_writer_get_feature_string. The three XML writer features added are \fBRAPTOR_FEATURE_WRITER_AUTO_INDENT\fR with boolean value (default true) to auto-indent the XML, \fBRAPTOR_FEATURE_WRITER_AUTO_EMPTY\fR with boolean value (default true) to automatically generate empty elements if a start/end element sequence has no content and \fBRAPTOR_FEATURE_WRITER_INDENT_WIDTH\fR with an integer value (default 2) to set the indenting level for the XML.

New build configuration and portability fixes for win32 (John Barstow)

Portability fixes for win32 - added SIZEOF_UNSIGNED_SHORT (Dave Viner, others)

Added a signing memory debugging system to aid checking when raptor-allocated memory is freed in another library or vice-versa enabled by --with-memory-signing configure option (defaults to on in maintainer mode).

Fixed a few internal malloc/frees to use RAPTOR_MALLOC / RAPTOR_FREE so that the above signed memory system worked.

RDF/XML serializer: Use the maximal name when splitting a predicate.
Turn datatyped literals that are rdf:XMLLiteral into inline XML with rdf:parseType="Literal" rather than XML-escaped.

RDF/XML abbreviated serializer: Fix a crash when there is a NULL base URI. Use the maximal name when splitting a predicate.
Turn datatyped literals that are rdf:XMLLiteral into inline XML with rdf:parseType="Literal" rather than XML-escaped.

RSS tag soup parser: Fix crash with unexpected use of alternate attribute.
Update from Suzan Foster to reflect the latest status of the enclosure vocabulary and allow multiple common items and fields.

RSS 1.0 serializer: Added RSS enclosures serializing.

grapper example GTK program now stores the window width and height using gconf2.

Raptor 1.4.5 Changes

Added a new RDF/XML with abbreviations serializer rdfxml-abbrev written by Steve Shepard which handles several of the abbreviations specified by the RDF/XML Syntax Specification (Revised) W3C Recommendation. It is suitable for writing small documents as there are known scaling issues.

The RSS tag soup parser was updated to work better when there is no base URI given. It also now supports reading the RSS 1.1 format and turning it into RSS 1.0 model triples.

Deprecated raptor_ntriples_string_as_utf8_string as rather too internal to be useful, since it only works with a parser.

More fixes to work around the broken libxml2 on Apple OSX 10.3.x with inconsistent shared libraries / headers.

Experimental and incomplete Notation 3 parser - updated to match changes to Turtle. CVS changes only, not enabled in standard builds.

Raptor 1.4.4 Changes

Make the RSS tag soup parser handle RSS 0.9 namespace elements by turning them into RSS 1.0.

Fix a couple of crashes in the RSS 1.0 serialiser when no base URI is used.

Make raptor_uri_to_relative_counted_uri_string work when the base or reference URI have no paths such as like http://example.org

Added portability fixes for Win32 to get Raptor 1.4.3 building with MS Visual Studio using expat and libcurl. The RAPTOR_INTERNAL define was moved to the build configuration and defines added for integral type sizes. Patch from Dave Viner (dviner at apache dot org).

Raptor 1.4.3 Changes

A release with the major new feature of an XML writer API. This is now used along with a new supporting XML element class to improve the existing RDF/XML serializer and to provide a new RSS 1.0 serializer.

This API it is also used by the next release of Rasqal to provide serializing of query results to XML.

The new raptor_xml_writer class functions added are: raptor_new_xml_writer (constructor), raptor_free_xml_writer (destructor), raptor_xml_writer_empty_element, raptor_xml_writer_start_element, raptor_xml_writer_end_element, raptor_xml_writer_cdata, raptor_xml_writer_cdata_counted, raptor_xml_writer_raw, raptor_xml_writer_raw_counted, raptor_xml_writer_comment and raptor_xml_writer_comment_counted.

The new raptor_xml_element class functions added are: raptor_new_xml_element (constructor), raptor_free_xml_element (destructor), raptor_xml_element_get_name, raptor_xml_element_set_attributes, raptor_xml_element_declare_namespace and raptor_iostream_write_xml_element.

Parser Changes

RSS tag soup parser now works with older libxml2s (2.5.10+), including the one shipped with some Apple OSX versions that has an inconsistent header file and library.

RSS tag soup parser recognises/scores more common XML RSS file names.

RSS tag soup parser turns XML RSS <guid isPermaLink="true">val</guid> into RDF/XML form <guid rdf:resource="val"/>, leaving the non isPermaLink form to be a literal value.

A bug was found in libxml2 that causes double expanding of XML entities in RDF/XML. This has been reported but cannot be worked around from raptor. The expat XML parser can be used as an alternative, as it does not have this problem. A test was added for this bug but it will not cause the test suite ('make check') to fail.

Added additional Turtle parser tests that cover Notation 3 syntax that is not part of the Turtle language.

Added raptor_parser_set_feature_string and raptor_parser_get_feature_string methods to set/get string feature values.

Serializer Changes

Added feature relative_uris for serializers. This is used by the RDF/XML serializer and enabled by default.

Added feature start_uri for serializers with a string value to set the start URI for serializing. Not used at present.

Added new methods raptor_serializer_features_enumerate to list serializer features and functions to set/get serializer feature integer or strings values: raptor_serializer_set_feature, raptor_serializer_get_feature, raptor_serializer_set_feature_string and raptor_serializer_get_feature_string.

Added raptor_serialize_set_namespace to allow user declaration of prefix/URI namespaces pairs as serializing hints.

the RDF/XML serializer was improved using the new XML Writer class so it now uses any user-declared namespace hints in it's output and emits relative URIs whenever possible. The latter was provided by a patch from René Puls.

A new RSS 1.0 serializer was added, using the new XML Writer class and using the same structures, classes and properties as the RSS tag soup parser.

URI class changes

Added relative URI generating code from a patch written by René Puls and provide this with two new methods raptor_uri_to_relative_uri_string and raptor_uri_to_relative_counted_uri_string.

Added raptor_uri_print to print a URI to a file handle.

Added methods raptor_uri_to_string and raptor_uri_to_counted_string to return a URI as newly allocated strings.

I/O Stream Changes

Many classes gained methods to write to iostreams, supporting the new XML Writer class functionality. The added methods are: raptor_iostream_write_namespace, raptor_iostream_write_ntriples_string, raptor_iostream_write_qname, raptor_iostream_write_statement_ntriples, raptor_iostream_write_stringbuffer, raptor_iostream_write_xml_element and raptor_iostream_write_xml_escaped_string.

Namespace Class Changes

Added raptor_namespace_copy copy constructor and raptor_new_namespace_from_uri constructor to build a namespace from a raptor_uri object.

Added utility function raptor_new_namespace_parts_from_string to decode syntax of the form xmlns:prefix="uri" into prefix and uri string pairs.

Added raptor_namespaces_find_namespace_by_uri method for namespace stack to find a declared namespace by URI. This complements raptor_namespaces_find_namespace which already provides searching by prefix.

Unicode and UTF-8 Changes

Added several methods for checking characters forming parts of XML 1.0 or XML 1.1 names: raptor_unicode_is_xml10_namestartchar, raptor_unicode_is_xml11_namestartchar, raptor_unicode_is_xml10_namechar and raptor_unicode_is_xml11_namechar.

Added a function raptor_utf8_check to check that a string is legal UTF-8 and all the encoded Unicode characters are in the range U+0 <= character <= U+10FFFF

Added a function raptor_xml_name_check to check that a string is a legal XML name (1.0 or 1.1) as well as legal UTF-8.

Other Changes

Feature support: Added raptor_feature_value_type to determine value of a feature - either integer (most) or string.

XML QName class: Added raptor_qname_copy copy constructor.

Sequence class: Added raptor_sequence_join to join two sequences of items, leaving one empty.

Statement class: Added raptor_statement_copy copy constructor and raptor_free_statement destructor. Previously these were internal to raptor.

The rapper utility was modified to add a feature form: -f xmlns:PREFIX="URI" allowing the setting of output serializer namespaces.

The namespace URI string constants exported by raptor are now of type unsigned char*.

Raptor 1.4.2 Changes

Make raptor_xml_escape_string fail correctly when given bad UTF-8 to escape.

Raptor 1.4.1 Changes

Fixed a buffer overrun in decoding a URI scheme in raptor_uri constructors such as raptor_new_uri.

Fixed a crash in RSS enclosures crash when the url attribute seen on a non-<enclosure> element

raptor_xml_escape_string return value has changed to be an int, returning <0 on failure. This allows the empty string encoding an empty string case to work and be distinguished from an error.

Raptor 1.4.0 Changes

A release with the major new feature of providing serializing of RDF triples to syntaxes. It also added a new support class for I/O streams and had other minor fixes.

Added a Raptor Serializer class (raptor_serializer) with similar style to Parser (raptor_parser). Two serializers are provided, for RDF/XML and N-Triples. The serializing can be done to files, C FILE* or to strings. The raptor_iostream class that provides this also allows writing to any other form by creating a custom iostream.

The new raptor_serializer class functions added are: raptor_serializers_enumerate, raptor_serializer_syntax_name_check, raptor_new_serializer, raptor_free_serializer, raptor_serialize_start, raptor_serialize_start_to_filename, raptor_serialize_start_to_string, raptor_serialize_start_to_file_handle, raptor_serialize_statement, raptor_serialize_end, raptor_serializer_get_iostream, raptor_serializer_set_error_handler, raptor_serializer_set_warning_handler and raptor_serializer_get_locator

Added a Raptor I/O stream abstraction in raptor_iostream class to support serializing of RDF to multiple output streams such as to filenames, to C standard I/O FILE* handles and to strings especially for cross-language use. A raptor_iostream_handler can be used to construct a user-defined iostream.

The new raptor_iostream class functions added are: raptor_new_iostream_from_handler, raptor_new_iostream_to_sink, raptor_new_iostream_to_filename, raptor_new_iostream_to_file_handle, raptor_new_iostream_to_string, raptor_free_iostream, raptor_iostream_write_bytes, raptor_iostream_write_byte, raptor_iostream_write_end, raptor_iostream_write_string, raptor_iostream_write_counted_string, raptor_iostream_get_bytes_written_count, raptor_iostream_write_decimal and raptor_iostream_format_hexadecimal.

The rapper utility was modified to use serializer class so that the output formats supported are now N-Triples (-o ntriples) - the default, and RDF/XML (-o rdfxml).

Raptor now exports more static namespace URI strings for general application use: raptor_xml_namespace_uri, raptor_rdf_namespace_uri, raptor_rdf_schema_namespace_uri, raptor_xmlschema_datatypes_namespace_uri, raptor_owl_namespace_uri, and the length raptor_rdf_namespace_uri_len.

The raptor_stringbuffer class gained a new method raptor_stringbuffer_copy_to_string which allows efficient copy-out of a constructed string.

The raptor_www class gained a new method raptor_www_fetch_to_string to allow retrieving of web content as a single string.

RSS tag soup parser gained support for generating triples for enclosures, after a patch from Suzan Foster. Changes made include correcting the enclosures namespace and tidying some memory leaks.

Raptor 1.3.3 Changes

A release with major improvements along with several minor fixes.

Raptor's License was changed from LGPL 2.1/MPL 1.1 to LGPL 2.1/Apache 2

Thanks to Chris Pointon for several patches to make Raptor easier to build under Win32 which were applied, with some slight modifications.

Increased WWW content retrieval buffer size from 256 bytes to 4K since this was causing problems for even moderate size documents.

After testing raptor on a very large RDF/XML file with many rdf:ID values, the check for duplicate values was found to be inefficient in memory and slow. The implementation was improved to be more memory efficient and a new parser feature check_rdf_id was added to disable checking (default is enabled).

Added a new Unicode NFC checker to replace the functionality formally available by calling the GNOME glib function g_utf8_normalize. This new checker is done via several tables and adds approximately 50K to the object size of the library when compiled on x86. This code and tables can be disabled with configure option --disable-nfc-check causing all checks to succeed.

Fix the exporting of raptor_xml_literal_datatype_uri_string and raptor_xml_literal_datatype_uri_string_len as constants for use by applications. Previously raptor.h wasn't doing this correctly.

Added raptor_calloc_memory for allocating zeroed memory inside raptor, for use by applications passing memory in/out of raptor.

Added a new configure option --enable-parsers to allow the selection of the required RDF parsers from any of those supported (RDF/XML, Turtle, N-Triples, RSS tag soup).

Reorganised the sources to split parsing support from RDF/XML to support compiling without this parser.

Updated the RSS Tag Soup parser to start to handle the Atom 0.3 currently being standardised by the IETF Atom Publishing Format and Protocol working group.

Altered the Turtle parser to work with large source documents that exceeded bison limits. Thanks to Geoff Chappell for providing a fix for this.

Rewrote the URI parsing to create an internal structure and improved the relative URI resolving in preparation for future work such as potentially supporting URI canonicalisation such as proposed to be used by Atom.

Raptor 1.3.2 Changes

A release with some minor fixes.

Added a new configure option --with-expat-source=DIR to allow the use of external expat source trees in either the old or newer directory structure style. (Patch from Mark Smith).

Added raptor_alloc_memory for handlers that need to allocate memory in the same heap as raptor uses for raptor_free_memory. This is mostly useful for allocating memory that is freed by raptor in error, ID and statement handlers on win32 which has separate heaps for different DLLs.

A bug was fixed where errors which happened when fetching WWW content were always printed to stderr. They are now passed to the main error routines which allows applications to retrieve them.

Accessor functions were added for parts of the public raptor_locator structure which makes it possible to get structured error information from language bindings via Redland (Patch from Edd Dumbill). The new functions are:

The Unicode Normal Form C (NFC) checking via the GNOME glib library function g_utf8_normalize is broken, comparing the data it says is failed against other NFC checkers. It is also slower than need be since it is doing full normalizing rather than just checking for NFC, and adds a rather large dependency for just one function. A new portable checker will be added in a later release.

Raptor 1.3.1 Changes

A release primarily to fix some win32 and portability issues.

raptor.h now includes stdarg.h

Corrected the raptor_print_statement declaration in raptor.h for the argument statement to have one less 'const' which matches the actual code.

Made several portability fixes for compiling natively on win32 which doesn't quite do POSIX or C99.

Changed the support for file: URIs and converting to and from filenames. It now %-escapes spaces and % characters on conversion to and from filenames with raptor_uri_uri_string_to_filename, raptor_uri_uri_string_to_filename_fragment and raptor_uri_filename_to_uri_string. For Win32, more tests were added and the format of URIs supported corrected to use the file:///c: form rather than file://c|/

URIs that resolve to directories now return an error when lstat is available to check.

Parser Changes

The Turtle parser was updated to only allow language with non-datatyped literals, allow a '_' immediately after a ':' in qnames and to make a bare ':' qname work correctly.

The Turtle parser was fixed to re-initialise correctly when performing multiple parsings. The other parsers already did this correctly.

Added a warning to the RDF/XML parser for unknown rdf:parseType values, when parsing in lax mode - which is the default. It now tells the user when the parsing is working as 'Literal' mode by finding an unknown value. This is controlled by a new parser feature warn_other_parsetypes which is default set true in lax mode. Parser modes are controlled by the raptor_set_parser_strict method.

Raptor 1.3.0 Changes

A release primarily to provide support for the new Rasqal RDF query library but with some new features and fixes.

Parser Changes

Added a new constructor raptor_new_parser_for_content to guess the parser to use from hints in URIs or content, using a new utility function raptor_guess_parser_name.

Additional checks were added to the RDF/XML parser for RDF-namespaced names in element and attributes and if they are forbidden giving an error otherwise if unknown, giving a warning.

The Turtle parser was updated to correct the collections syntax, allow '-' in names and QNames and to add integer literals. This parser now correctly uses raptor_generate_id when a blank identifier name is needed.

Completed parser feature support by adding raptor_get_feature, raptor_feature_from_uri, and raptor_features_enumerate to get values and enable discovery of supported features at run time. raptor_set_feature was changed to give return a success value

Added a new method raptor_get_mime_type to get the MIME type of the syntax for a parser

raptor_parse_uri_with_connection (which is called by raptor_parse_uri) now sets the HTTP Accept: header to the MIME type of the parser in WWW requests using the new raptor_www_set_http_accept.

rapper changes

Added options -f/--feature for setting features and -g/--guess for guessing syntax from some content or identifiers. See rapper(1) for all rapper options.

Utility function changes

Added raptor_syntax_name_check to check for valid syntax language names.

Added raptor_free_memory to free memory returned by raptor functions.

Added Unicode utility functions raptor_unicode_char_to_utf8 and raptor_utf8_to_unicode_char.

Exported URI string raptor_xml_literal_datatype_uri_string.

Deprecated raptor_print_statement_detailed always intended to be internal.

WWW Class changes

Added support to set the HTTP Accept: header for curl and libxml2 when retrieving HTTP content by the new raptor_www_set_http_accept method.

New classes - Sequence and Stringbuffer

Added a utility class raptor_sequence providing simple sequences that can handle stacks and queues

Added a utility class raptor_stringbuffer for constructing strings from substrings appended or prepended.

Raptor 0.9.0 - Raptor 1.2.0 Changes

Release notes for 1.2.0 and earlier are in the NEWS page or ChangeLog


Copyright 2003-2005 Dave Beckett, Institute for Learning and Research Technology, University of Bristol