An abundance of software comparison tools is available for datasets where simply line by line is suitable, but few tools can understand change in structured data like JSON or XML. Understanding how content or data is structured, requires more sophisticated algorithms and representation, allowing for accurate representation of actual change. This is important for use cases where you are receiving big data sets where change is impossible to represent without structurally aware analysis, and in the provision of data as a delta or set of changes rather than repeatedly issuing huge datasets.
Pandemic data sets are usually very large as they often represent national scale information, with daily data sets for infection rates, cases, deaths, and vaccinations. This requires data analysts to compare data with their current model to establish what changed and update relevant data. This level of processing can be time consuming, reducing immediacy and accuracy of reporting with subsequent analysis and decision-making being delayed. A more appropriate and expedient approach would be to process a delta or difference file to compare the current data set with the previous being more accurate and updating your data model far more quickly.
Publishing a delta or difference file of changes makes more sense, and should perhaps be more widely adopted, so why hasn’t this been the case? Often the answer is that producing a reliable and accurate delta file of changes if not straightforward. There is no standard for representing change in CSV, and whilst JSON has patch it is deficient in many respects for this purpose. However, where data is structured, it can be done successfully and DeltaXML’s patented delta file has been doing this for several years with XML and JSON.
One of the significant benefits of publishing changes is being able to publish changes between any two arbitrary files, allowing change to be derived over extended ranges. An example would be the International Organization for Standardization (ISO) who make available the latest version of a standard with changes from a selected previous version marked up. DeltaXML technology provides the technology behind this, to allow changes to be marked up on demand.
The pandemic has made the Pandora’s box of government data transparent, open, accessible, and useable to netizens in every country. When change happens, it matters, and tools like those from DeltaXML are helping organisations around the world to be at the forefront of understanding change in structured content and data. There is responsibility, now more than ever, that data is not misrepresented or misunderstood, to ensure credibility and authority of data sources such as public institutions.
UK Government data source for daily data sets for infection rates, cases, deaths, and vaccinations – https://coronavirus.data.gov.uk/
Sign up to the DeltaXML newsletter to get the latest on DeltaXML and other company news.
WHERE CHANGE MATTERS
From ISO Quality Services Ltd.
Comparison analysis and opportunities for future improvements – DDD #...
Scroll to top
