Original URL: http://www.theregister.co.uk/2006/01/09/java_xml_filter_pipeline/

Hands on with Java XML filter pipelines

Ignored by many

By John Hunt

Posted in Developer, 9th January 2006 15:42 GMT

In this tutorial we will step through this little understood area of XML processing. Java XML Filters are part of the SAX API for XML processing.

Java XML filters are objects that can filter (or process) data within a pipeline of filters, wrapped around a core XML processing application. The SAX API is a standard part of the JAXP (Java API for XML Processing), which has itself been provided as part of the Java environment since Java 2 SDK 1.4.0 and can be used in a wide range of XML processing applications.

In the remainder of this tutorial we will first introduce the concept of Java XML filters and examine how a generic Java XML Filter can be written.

The XMLFilter interface

Although the XMLFilter interface is part of the SAX API for processing XML documents, it's ignored by many Java XML tutorials and overlooked by most Java developers. An XMLFilter is a sub-interface of the XMLReader class; as such it is very like the XMLReader except that it obtains its events from another XML reader rather than a primary source like an XML document, file or database. As such, it is a primary component in the JAXP pipeline architecture. That is, it can sit within a pipeline, receiving XML data from another XML processing element and passing the results of its processing onto another XML processing element if required.

Assuming you have a distribution of SAX, or a version of Java containing the XML APIs, you can look at the included classes; the one you want is org.xml.sax.XMLFilter. You should also ensure that you have the SAX helper classes, found in the org.xml.sax.helpers; package. In that package, you will want to focus on the org.xml.sax.helpers.XMLFilterImpl class.

The XMLFilter interface methods

If you examine the XMLFilter, you'll find that it extends the org.xml.sax.XMLReader interface, and adds two new methods:

1. public void setParent(XMLReader parent); this method allows the application to link the filter to a parent reader (which may be another filter). The argument may not be null.

2. public XMLReader getParent(); this method allows the application to query the parent reader (which may be another filter). It is generally a bad idea to perform any operations on the parent reader directly: they should all pass through this filter.

This probably doesn't look like much; however it is very significant as the ability to link one filter to another (via the setParent method) allows the "pipeline" to be constructed. Of course, you also get all the other XMLReader methods such as startElement(), endElement(), etc. In each of these methods, you can operate upon the input XML data before an application, or the next filter in the pipeline, gets to it.

This means that in the following diagram you get to modify data before an application gets that data using a standard XML framework. Note that as you have all of the SAX callback methods that an XMLReader does, you can work with the elements, the attributes, the prefix mappings, and anything else that SAX can work with.

Filter data for an XML Application

The key here is that the XML form the source, can be filtered (pre-processed) before being accessed by the application, without modification to that application, and in a standard manner. Essentially, the first filter reads the XML data, processes it if necessary and pushes it onto the next filter. Filter 3 then does the same and pushes it onto the XML application. Of course you could do the same thing by writing your own custom XML pre-processors, the point here is that a standard framework is available within which to do this.

Typical uses of Filters

Some typical uses of filters include:

• Normalization of whitespace in which contiguous whitespace PCDATA is replaced by a single space.

• Ignoring information in the originating XML document.

• Modifying elements in the original XML document.

• Adding data to the elements in the original XML document.

The following example illustrates how a filter can be used to modify an actual element such that the element passed to the processing application differs form that define din the original XML document.

A simple filter

The following listing shows a very simple SAX filter that changes all postal code elements into postcode elements. This effectively pre-processes the input document to modify one element, while allowing all other elements to pass through unchanged. This filter is presented in the following listing.

A Simple XMLFilter class

To be a Java XML filter a class must either implement the XMLFilter interface or extend the XMLFilterImpl class. In our case we are extending the XMLFilterImpl class from the org.xml.sax.helpers package as this means that we only need to implement those methods that will actually do something, (all other methods required by the XMLFilter interface are provided by inheritance from the XMLFilterImpl class). This keeps our code cleaner, and requires less work on our part.

We now free to implement only the methods startElement and endElement (be careful to make sure that the method signatures are the same as those in the XMLReader interface, otherwise you will be overloading the methods rather than overriding them, which will mean that your code will not be called).

Our startElement and endElement methods change the localName and the qName (qualified name) of the element to from the American postal code to the British postcode if the element postal code is found. Otherwise they just pass the data through unaltered.

Be careful to call the inherited super class methods once you have finished processing your data. This will enable the pipelining behaviour to be invoked.

SAX Processing Rules

You may wonder why we set both values. This has to do with rules regarding SAX processing and the state of the following two SAXParserFactory properties:

http://xml.org/sax/features/namespaces and the

http://xml.org/sax/features/namespace-prefixes properties.

Essentially, these rules say that:

1. the Namespace URI and local name are required when the namespaces property is true (the default), and are optional when the namespaces property is false (if one is specified, both must be);

2. the qualified name is required when the namespace-prefixes property is true, and is optional when the namespace-prefixes property is false (the default).

To handle these situations we are setting both parameters to the new element name. This is also why we test both parameters.

Setting up a pipeline

Once you have your filter set up and compiled, you need to create a pipeline for processing your XML. This should move from input XML document to the filter to the reader. You may even have multiple filters, stacked upon each other. As long as input comes first, and your reader (with application-specific callbacks) comes last, things work fine.

The following listings show how to set up a program to use filters. To do this we have defined a simple XML application (called SimpleXMLApplication). This is a standard SAX Content Handler (it extends the DefaultHandler to obtain the default behaviour). This application merely echoes, to the standard output, the XML passed to it (with suitable indentation).

sax content handler

The XMLFilterTest test harness class links the SimpleXMLFilter and the SimpleXMLAPplication together. Notice that because the one or more filters must sit between input source and the reader, all the operations that you would normally invoke on the reader are invoked on the filter. It then delegates any data that passes through the filter to the reader.

p>java pipeline simple application XMLFilterTest

Also note that we obtain the XMLReader form the root parse and set that as the "parent" of the filter. We then link the filter with the application by making the application the content and document handler of the filter.

The effect of running the XMLFilterTest on a simple XML document is presented below:

effect of running the XMLFilterTest

Data Pollution

However, a word of warning if you use a filter to remove some elements form the data input to the next element in the pipeline. It is all too easy to pollute the data being sent on. For example, consider the case where you don't delegate in the startElement() method for certain data, but forget to do the same in endElement(). The result would be that some elements would never be reported as starting, but would be reported to the reader as ending. This would cause, in the best case, program errors, and in the worst case, data loss or corruption in your application.

Real world use

As an example of a tool that makes extensive use of pipelines of filters, consider the DeltaXML XML diff tool. This uses XML filters to pre and post process XML data before performing XML comparisons, synchronization operations and patches. To download a time limited copy of DeltaXML see the DeltaXML web site here. ®