We have something of a chicken-n-egg problem here,  we only mention Java on our website, some people do contact us asking if we have a .NET product, but how many move onto the next hit in their google list?  This forum message attempts to, at least partly, solve this problem.

The good news is that we have done a number of prototypes using IKVM.net to translate Java byte-code into the CLR equivalent.  It works quite well!  We've also put some effort into resolving some API differences between the JDK APIs (Sun, Apple, IBM) and GNU-Classpath.

At the moment the command-line tools work and you can call the translated DeltaXML and JAXP classes from languages such as C#.  This works well if you have a knowledge of both plaforms.

We would like some user-feedback:

  * please tell us if you need .NET - we need to prioritize our development efforts

  * How do you store your XML data on .NET?  We need to look at some API wrapping/interfacing between the .NET classes and our existing Java classes.  Please tell us which parts of System.Xml you currently use.

  * We currently write fast/memory-efficient filters using the SAX XMLFilterImpl class.  Do you want an equivalent for .NET?  Perhap 'wrapped' XmlReaders?

Please do leave us some feedback, either here, or by email.

Cheers,

Nigel

Thanks for the detailed response.  I understand the complexity and accept your logic of where the correct location to perform this action belongs (post filter or application processing logic). 

However, I have a problem finding a solution using these methods.  Here is a more detailed description of what I am attempting to accomplish.  It may be trivial to accomplish in an output filter, but I haven't been able to come up with a simple answer yet.

Here is an example of the XML we are handling (simplified for this purpose):

<example>   <object nubmer="1" UID="abc">     <property name="test">value1</property>   </object>   <object nubmer="2" UID="def">     <property name="test">value2</property>   </object>   <object nubmer="3" UID="ghi">     <property name="test">value3</property>   </object> </example>

The second document may insert an "object" between current object 1 and 2.  Subsequently, the "numbers" will be different.  See below:

<example>   <object nubmer="1" UID="abc">     <property name="test">value1</property>   </object>   <object nubmer="2" UID="jkl">     <property name="test">value2</property>   </object>   <object nubmer="3" UID="def">     <property name="test">value3</property>   </object>   <object nubmer="4" UID="ghi">     <property name="test">value3</property>   </object> </example>

Using the "UID" attribute as the key (defined in the input filter - not shown below), will ensure we properly match the objects.  If I use a post filter to remove the extraneous modification information on the number attribute, I will still be left with a deltaxml:delta attribute on the object.  My issue here is that my application processing relies on this information and displays to the user that the object is modified based on this attribute.  However, the only modification is the number and I will remove that modification.  This will leave the display indicating there is a change and there will be no change visible.

<example xmlns:deltaxml="http://www.deltaxml.com/ns/well-formed-delta-v1" deltaxml:delta="WFmodify">   <object deltaxml:delta="unchanged" nubmer="1" UID="abc">     <property name="test">value1</property>   </object>   <object deltaxml:delta="add" nubmer="2" UID="jkl">     <property name="test">value4</property>   </object>   <object deltaxml:delta="WFmodify" UID="def" deltaxml:old-attributes="nubmer=2" deltaxml:new-attributes="nubmer=3">     <property deltaxml:delta="unchanged" name="test">value2</property>   </object>   <object deltaxml:delta="WFmodify" UID="ghi" deltaxml:old-attributes="nubmer=3" deltaxml:new-attributes="nubmer=4">     <property deltaxml:delta="unchanged" name="test">value3</property>   </object> </example>

As I type this, it is very apparent that there is a simple solution to my "problem".  The solution is to look at all child elements of the object and if all child nodes are unchanged, remove the deltaxml:delta attribute on the object.

Sorry for the long message with now no apparent purpose.  I left the detail just in case someone else might find it helpful.

Thanks for you help and clarification.

Jody

Hi Jody,

This is almost becoming an FAQ - since this is the first time is has appeared as a forum message - I'll take the oportunity for a detailed (but long - sorry!) response:

When users ask for ways to "ignore attributes" or "ignore attribute changes", they also usually add a priviso, such as "but they must appear in the result"

When we see this type of question:

It is critical that the data remain available for later evaluation, but it is data that is expected to change and therefore should not be compared.

We often respond by asking a question based on this small example:

<x a='1' b='2' d='4a'/>    compared-with    <x a='1' c='3' d='4b'/>

In this example, the 'd' attribute changes value, so which value should be in the result?   You asked for it not to be compared, but you need a data value for later evaluation; which value should it be?  Similarly if a value is deleted (such as the 'b' attribute which doesn't appear in the 2nd input) do you want to know about it?

For the above example, we currently produce a delta that looks like this:

<x a="1" deltaxml:delta="WFmodify"    deltaxml:new-attributes="c='3' d='4b'"    deltaxml:old-attributes="b='2' d='4a'" />

Admittedly you've got to do some work to see that 'd' has changed in this delta format (and this is something we will fix in our new delta format), but this is a complete description of the difference.

If you wanted to ignore attributes, and we wanted a fairly general solution for all customers, we would have a number of configuration options/questions:

1.  If an attribute appears with the same value in both inputs should it be present in the result?  (we currently handle this case with the changes-only vs. full-context delta option).

2.  If an attribute is only present in the first (old) input, should it appear in the result?

3.  If an attribute is only present in the second (new) input, should it appear in the result?

4a.  If an attribute is present in both inputs, but with different values should that attribute appear in the result?

4b.  If the answer to 4a is yes, should the value be the first/old value, the second/new value, or some other value?


Rather than have a complicated interface on the comparator we would argue that it is better to process/extract the information from the delta and keep the comparator as it is.

Here are some additional reasons:

* We like deltas that are provably correct - we can take a minimal delta and use the (re)combine operation and generate one input from the other, we can also generate either input from a full-context delta.    If we loose this property by adding some options it makes it harder to test the comparator.

* as the questions above demonstrate the comparator interface  could be complicated, add in your 'specific set of elements' requirement and it is even more so.

* an output filter could be more powerful, and could use logic based on the full power of XPath and also be based on other (parent/child/sibling) change information/results if done post-comparison.

So going back to your request:

I am looking for an effective way to ignore an attribute on a specific set of elements.  It is not dissimilar to the date example at http://www.deltaxml.com/dxml/library/how-to-treat-timestamps.html.  It is critical that the data remain available for later evaluation, but it is data that is expected to change and therefore should not be compared.

We would suggest that you should compare everything and generate the delta file in the normal way, but then ignore or filter-out changes in the delta which you are not interested in. 

It is possible to write an 'ignore these changes filter' and slot it into your output filter chain.  If this filter does an 'undo changes' operation, then it could be coded/tested independently of any other output filters you are using.

A good starting point for XSLT filters is the deltaxml-folding-html.xsl filter in the samples directory, this has logic for dealing with the different types of attribute change.

Java filters are also possible (but slightly less powerful);   we have some prototype code, but we put this on hold pending our new delta format.

Do you have an example of what you want to do?  Perhaps
based on the element/attribute names, namespaced-names or XPaths?  We could code-up a remove attribute changes example for you.

One final point, such a filter would have to have some knowledge of the names or XPaths of the elements/attributes involved;  it could be used in all pipelines for all data, and thereby resolve the code paths issue:

would prefer to not have separate code paths based on which documents are being compared

To sum up:
- we think its better to process the delta
- we'll make this alot easier with our new delta format
- we'll help with a filter to undo the changes that are causing problems

Cheers,

Nigel

I am looking for an effective way to ignore an attribute on a specific set of elements.  It is not dissimilar to the date example at http://www.deltaxml.com/dxml/library/how-to-treat-timestamps.html.  It is critical that the data remain available for later evaluation, but it is data that is expected to change and therefore should not be compared.

Although I understand the option discussed in the example above, it is very impractical for our particular use.  The first problem is that it requires special handling of these specific documents (would prefer to not have separate code paths based on which documents are being compared).  Second, in our case, these documents are fairly large and the prospect of comparing them twice as a solution is unappealing.

Is there any discussion of a potential option to set a flag to ignore certain elements or attributes?

Thanks,

Jody

Hi Jody,

As Nigel mentioned, we have been talking about this very thing recently. After writing the XSLT version of this filter some time ago, we decided it would be too difficult/complicated to write a Java filter to achieve the same purpose. This decision was based on the fact that we were trying to recreate the 'match' statements found in the XSLT, not always a good plan when designing a Java version simply because of the different approach that is necessary when manipulating an event stream (as Java filters do).

After discussing this in a meeting this week, we realised that if we took another approach, the Java filter was actually fairly simple. It has been written and will be included in the next release of Core. However, if you wish to use it now I have included the source below.

Thanks,

Tristan

import org.xml.sax.Attributes; import org.xml.sax.SAXException; import org.xml.sax.helpers.AttributesImpl; import org.xml.sax.helpers.XMLFilterImpl; public class RemoveExchange extends XMLFilterImpl {   private final String DXML_NS= "http://www.deltaxml.com/ns/well-formed-delta-v1";   private final String DXML_OLD= "old";   private final String DXML_NEW= "new";   private final String DXML_EXCHANGE= "exchange";     private boolean saveText= false;     private String savedText= "";   public void startElement(String uri, String localName, String qName, Attributes atts)     throws SAXException   {         if (uri.equals(DXML_NS) && localName.equals(DXML_EXCHANGE)) {       // don't call super.startElement - this element is removed     } else if (uri.equals(DXML_NS) && localName.equals(DXML_OLD)) {       saveText= true;     } else if (uri.equals(DXML_NS) && localName.equals(DXML_NEW)) {       saveText= true;     } else {       // a deltaxml:old or deltaxml:new can either contain text or a single element, if       // we find another element we cannot have any text, so saveText is made false       saveText= false;       super.startElement(uri, localName, qName, atts);     }   }     public void endElement(String uri, String localName, String qName) throws SAXException {     if (uri.equals(DXML_NS) && localName.equals(DXML_EXCHANGE)) {       // we don't output the exchange close tag either     } else if (uri.equals(DXML_NS) && localName.equals(DXML_OLD)) {       if (saveText) {         //if we were saving text, we now need to output it in a PCDATAmodify element         super.startElement(DXML_NS, "PCDATAmodify", "deltaxml:PCDATAmodify", new AttributesImpl());         super.startElement(DXML_NS, "PCDATAold", "deltaxml:PCDATAold", new AttributesImpl());         super.characters(savedText.toCharArray(), 0, savedText.length());         super.endElement(DXML_NS, "PCDATAold", "deltaxml:PCDATAold");         super.startElement(DXML_NS, "PCDATAnew", "deltaxml:PCDATAnew", new AttributesImpl());         super.endElement(DXML_NS, "PCDATAnew", "deltaxml:PCDATAnew");         super.endElement(DXML_NS, "PCDATAmodify", "deltaxml:PCDATAmodify");         savedText= "";       }     } else if (uri.equals(DXML_NS) && localName.equals(DXML_NEW)) {       if (saveText) {         //if we were saving text, we now need to output it in a PCDATAmodify element         super.startElement(DXML_NS, "PCDATAmodify", "deltaxml:PCDATAmodify", new AttributesImpl());         super.startElement(DXML_NS, "PCDATAold", "deltaxml:PCDATAold", new AttributesImpl());         super.endElement(DXML_NS, "PCDATAold", "deltaxml:PCDATAold");         super.startElement(DXML_NS, "PCDATAnew", "deltaxml:PCDATAnew", new AttributesImpl());         super.characters(savedText.toCharArray(), 0, savedText.length());         super.endElement(DXML_NS, "PCDATAnew", "deltaxml:PCDATAnew");         super.endElement(DXML_NS, "PCDATAmodify", "deltaxml:PCDATAmodify");         savedText= "";       }     } else {       super.endElement(uri, localName, qName);     }   }   public void characters(char [] ch, int start, int length) throws SAXException {     if (saveText) {       savedText= savedText.concat(new String(ch, start, length));     } else {       super.characters(ch, start, length);     }   }    }

Hi Jody,

I would like to see an option to not use the deltaxml:exchange  element in the delta file.

You are not the first customer who doesn't like dealing with exchanges.  As you note they are/were useful in some situations, but in others they just add complexity.

Firstly some good news (announced for the first time here!):  we are working on a new delta format (a 'delta v2').  The primary motivation is for ease of processing; we want to make it easier both ourselves and our users to write filters and converters.  One feature of the new delta format is the removal of exchanges.  This work is in progress and should be available in Core and Sync releases by the summer.

Perhaps a pre-filter setting similar to the ordered setting that could define that "exchange" should not be used.

A bit of design background:  We prefer to use pre-filters where it should/could affect parts of the data in the input trees and/or how the Comparator processes that data; for example keying specific elements or spliting/converting certain elements by name or XPath.  For global settings we have in the past followed the SAX/JAXP models of having setFeature or setProperty methods in the APIs/DXP.  (The assumption here was that our users would perhaps be familiar  with these APIs?)

I understand the rational behind the exchange, but in certain instances in our delta files, it simply doesn't make sense.  I also understand there is post-processing filtering we can do to remove the exchange elements, but this extra processing on very large documents is not a preferred solution.

This is very timely, we were discussing this very problem (removing exchanges in large documents and XSLT vs Java filters) in our development meeting yesterday.
Stay tuned - we may have an interim solution (before the new delta format ) quite soon...

Cheers,

Nigel

I would like to see an option to not use the deltaxml:exchange  element in the delta file.  Perhaps a pre-filter setting similar to the ordered setting that could define that "exchange" should not be used.

I understand the rational behind the exchange, but in certain instances in our delta files, it simply doesn't make sense.  I also understand there is post-processing filtering we can do to remove the exchange elements, but this extra processing on very large documents is not a preferred solution.

Nigel, Chris, Tristan.  Great work on the new site.  Looking forward to using the new services.

Cheers,

Peter

A new release to support our new pricing and licensing models with the associated use of license keys and activation procedures.

Otherwise it is similar to the 4.2.x releases.

As always, comments, suggestions, bug-reports are always welcome.

Nigel

This (sticky) forum topic is intended for release announcements.

We normally also provide Core release notes online at the following URL:

http://www.deltaxml.com/core/current/docs/release-notes.html

The current part of the URL is rewritten to a version number, corresponding to the current release, for example: 4.3