1. Can we generate simple xml format as an output with simple flag of 'UPDATE', 'DELETE', or 'INSERT' rather than complex format with deltaxml:delta.

Yes, this can be done by processing the delta file, e.g. with XSLT. If by simple flag you mean an attribute, then I suggest you convert deltaxml:delta='add' to flag='insert'.

If you prefer to have an element, then convert as follows:

<para deltaxml:delta='add'>This is added</para>

to

<INSERT>   <para>This is added</para> </INSERT>

As the delta file is pure XML, you can convert it to whatever you want, even into SQL for example to generate a SQL update script direct from the delta.

2. If no, do you have already written parsers which could parse generated output files and create java objects, so that we can do further database operations based on flags and values.

There are standard tools available to do this, so we have not written any  ourselves. You can convert the XML delta file into a DOM, e.g. using W3C DOM, xom, dom4j; or you could use a JAXB like tool to create schema/dtd specific java objects.

Another possibility is to use an SQL extension to XSLT so that your database operations can be driven by XSLT. For example:
http://saxonica.com/documentation/sql-extension/intro.html

Let us know if you need more information on this.

Question by email re-directed to support forum:

Hi,

I have downloaded an evaluation copy of DeltaXML and impressed with the features. Could you please let us know –

   1. Can we generate simple xml format as an output with simple flag of ‘UPDATE’,’DELETE’, or ‘INSERT’ rather than complex format with deltaxml:delta.

   2. If no, do you have already written parsers which could parse generated output files and create java objects, so that we can do further database operations based on flags and values.

Thanks & Regards,

A new core release is available.

Highlights of this release include:
- Performance improvements with the combination of enhanced match 1 and keying
- a derivative matcher: enhanced match 2
- A new document-centric output filter designed to deal with mismatched common words such as 'and', 'the', 'is' etc. surrounded by differenced words.
- URIResolver and EntityResolver support in command.jar (useful for docbook processing)

See the release notes for further details/links:

http://www.deltaxml.com/core/4.4/docs/release-notes.html#r4.4

This release is now available for registered users on our website (evaluation and named-user/professional-server) keys
and to enterprise-server customers with support contracts via their usual download mechanisms.

One issue to be aware of: we fixed a bug (actually in 4.3.6) related to using DeltaXML in the Alfresco CMS - a clash of license-managers and Classloaders.  This fix unfortunately required a change to our activation/license key format.  Version 4.4 and later licenses will require a version 4.4 download ZIP or later.  For this upgrade, version 4.4 activation keys will not work with earlier downloads.

Cheers,

Nigel

Hi Nigel, thanks for your quick reply and your suggestions.

I've sent in the pre-processed data to support@deltaxml.com.

Thanks in advance for looking into this :-)

Cheers,
Philipp

Hello Philipp,

In my code, I've observed that deltaxml:key severly impacts performance.

That's a problem we would certainly like to take a look at.  Using keys should make things faster - The matching code when try to align two lists of sibling nodes at a particular level of the tree will use keys as a short-cut, it won't try things like exact subtree equality if it finds matching keys.

The situation is as follows: The XMLs to compare are approx. 4 MB each. Diffing takes about 45 sec. with raw documents but a whopping 50 minutes(!) when I add keys using a custom filter (takes about 5 seconds). Is this to be expected or due to an error on my side? I have already checked that the keys are unique (indeed they are). Also everything is ordered compare.

I can guess at one or two possibilities here:

- adding the filter causes the whole process/pipeline to nearly fill the available memory - most of the 50 minutes is taken up with garbage collection.

- The key values that you are using are somehow causing the hash functions in the java.util collections code to behave poorly.

I'm wondering however if nested assignment of the keys may have something to do with this... Is it recommended to use keys that way in a ordered comparison?

Keys can be nested, but don't need to be - the matching
process is recursive.
The rule is: keys must be unique amongst sibling elements of the same name or 'type' (equal local-name and NS URI)

I presume you've seen:  http://www.deltaxml.com/library/how-to-compare-orderless-elements.html
in particular section 4 (we should/must give this document a better title!)

Any hints are appreciated.

1. Do the filtering separately and then compare the filtered file - perhaps this is your 5s time?

2. Experiment with the heap size (java -Xmx ...)

3. Compare a file with itself - we don't do any shortcuts in this case, but it should be the optimal O(n) performance case.

4. If it is possible, please let us see the data or a small sample of it(either with the keys or the raw data with the keying code).
Our forum system doesn't support attachments, you could try pasting the data/code, or providing some URIs.  If your concerned about privacy you could also use email:   support@deltaxml.com

I'm curious/concerned about this problem (the keyed matching was code I wrote/tested) and I will be happy to help further.

Cheers,

Nigel

Hello,

In my code, I've observed that deltaxml:key severly impacts performance.

The situation is as follows:
The XMLs to compare are approx. 4 MB each. Diffing takes about 45 sec. with raw documents but a whopping 50 minutes(!) when I add keys using a custom filter (takes about 5 seconds).

Is this to be expected or due to an error on my side?

I have already checked that the keys are unique (indeed they are). Also everything is ordered compare.

I'm wondering however if nested assignment of the keys may have something to do with this... Is it recommended to use keys that way in a ordered comparison?

Any hints are appreciated.

Thanks, Philipp

Hello,

Thanks for the quick answer.
We're currently in the process of defining our needs and it may involve diff'ing and merging XML files.
There are a number of such tools available (both commercial and free software) which have different abilities and we are not sure yet which ones are required.

Your product is by far the most capable but we have to take the price into account, so I'll need the exact price when I'll make my report to chose between the different solutions.

I'll look forward to the answers you'll got from your meeting.

I must say that your web site is quite impressive, especially the demo available which help a lot to assess the product. But one thing you may improve is to have a permanent link to the pricing page : currently there is only a link from the main page (maybe I'm wrong ?). I've spent at least 10 mn to navigate through the different menu because I've missed the link when I first got on the main page...

Regards,

Its interesting to hear about your system:

... I liked the idea so much that I implemented a version myself that we use in several places.  I wanted to change things though so I could put in other transformations that can't be turned into an XMLFilter (e.g. I have Regex Transformations and EntityEncodingTransformations).  This allows all my clients to execute against a very simple interface, and I move the configuration of the entire pipeline, along with all the entity handling and reg ex transformations into a spring configuration.

I'll jot down some background and a few ideas about our future direction/plans in this area - it could be useful for people in similar situations and with or planning similar customizations:

The PipelinedComparator idea does have its limitations, it is an 80/20 type of design - we wanted to hide the JAXP pipeline construction complexity behind a simpler API for many of the common pipeline use-cases.  Long  time users may remember our Advanced.java JAXP sample code - we spent a lot of time explaining how to adapt/customize this code.   

One reason for being org.xml.sax.XMLFilter based is to keep a streaming architecture as far as possible - were we code Java filters as replacements for XSLT we get good memory footprint benefits;  these are often more significant than the CPU time benefits.  Converting the WordByWord filters from XSLT to Java provided some big benefits to users processing 100+ MB files with some of our pipelines.

There's one more feature I'd like to add to PipelinedComparator:  'Pull Parser' based filtering (a third type of input or output filter) - think of wrapped StAX or System.Xml.XmlReader based code.

Another idea we're considering is providing DeltaXML comparator components for other Pipeline systems.   These should provide a wider range of components/stages/tools than our simple model of chains of 'filters' - and perhaps something similar to your regexp transformers.  A NetKernel module is in progress which should allow the Comparator to be used in DPML pipelines and NetKernel more generally (its how we implement the sandboxes and services on this site).

We're also interested in the W3C XProc http://www.w3.org/TR/xproc/ standard and are considering providing a comparator step in one or more of the XProc implementations which are now becoming available.  I spent the weekend before last at XML Prague (papers and slides online:  http://www.xmlprague.cz/) where there was a lot of discussion on the topic of XML Pipelines.

Given all that, it didn't make sense for me to use the pipelined comparator, since I had my own version and I wanted to just use the base comparator you provide.

Yes I can appreciate that - we're still maintaining/supporting the base JAXP interface - so we will add this missing functionality at some point.  Customers with support contracts could influence the priority of this by asking through our enterprise support channels.

Cheers,

Nigel

Hello John,

I want to be able to do a 3 way merging, so I suppose that I need DeltaXML Sync. But on the pricing page, DeltaXML Sync is only offered as anadd-on to Core Enterprise, not Core Professional (and I haven't really understood the price either).

Yes, on reflection the wording could be improved.
We will get this improved over the next day or two.
Sync is not a core 'add on' but it stand-alone product.
It does 3 way change processing only.

I can't afford to buy the Core Enterprise, but I could buy the Core Professional, so would it be possible to purchase Core Professional + Sync ?

We do not currently have Sync Professional Named-User or Professional Server offerings;  I will bring
up your request with our sales and marketing people at a meeting
on Friday.  In the meantime Sync (and Core) evaluations are available;  if any
of the Named-User/Server limitations (such as node or CPU limits)
make your evaluation difficult let us know and we will
provide an enhanced activation key.

And would it be possible to have a better understandable pricing page (especially about Sync).

Yes - we'll improve the wording on the web page. 

Thanks again for the input,

Nigel

Hello,

I'm rather interested by purchasing DeltaXML but I'm rather confused by the pricing page.

I want to be able to do a 3 way merging, so I suppose that I need DeltaXML Sync. But on the pricing page, DeltaXML Sync is only offered as an add-on to Core Enterprise, not Core Professional (and I haven't really understood the price either).

I can't afford to buy the Core Enterprise, but I could buy the Core Professional, so would it be possible to purchase Core Professional + Sync ?

And would it be possible to have a better understandable pricing page (especially about Sync).

Thanks a lot,