Government Data, Where Change Matters

The pandemic has seen the public exposed to considerable data and statistics, and this has sparked the interest of many netizen projects and websites to make data more accessible. Daily updates from news channels can leave many feeling confused, but the release and publication of raw data is important for transparency and has set new standards for governments and public institutions. Now we have better access to data how can we make sense of the volume and velocity of changing data?

The value of understanding true XML and JSON change

An abundance of software comparison tools is available for datasets where simply line by line is suitable, but few tools can understand change in structured data like JSON or XML. Understanding how content or data is structured, requires more sophisticated algorithms and representation, allowing for accurate representation of actual change. This is important for use cases where you are receiving big data sets where change is impossible to represent without structurally aware analysis, and in the provision of data as a delta or set of changes rather than repeatedly issuing huge datasets.

Represent just the change

Pandemic data sets are usually very large as they often represent national scale information, with daily data sets for infection rates, cases, deaths, and vaccinations. This requires data analysts to compare data with their current model to establish what changed and update relevant data. This level of processing can be time consuming, reducing immediacy and accuracy of reporting with subsequent analysis and decision-making being delayed. A more appropriate and expedient approach would be to process a delta or difference file to compare the current data set with the previous being more accurate and updating your data model far more quickly.

Where change matters

Publishing a delta or difference file of changes makes more sense, and should perhaps be more widely adopted, so why hasn’t this been the case? Often the answer is that producing a reliable and accurate delta file of changes if not straightforward. There is no standard for representing change in CSV, and whilst JSON has patch it is deficient in many respects for this purpose. However, where data is structured, it can be done successfully and DeltaXML’s patented delta file has been doing this for several years with XML and JSON.

One of the significant benefits of publishing changes is being able to publish changes between any two arbitrary files, allowing change to be derived over extended ranges. An example would be the International Organization for Standardization (ISO) who make available the latest version of a standard with changes from a selected previous version marked up. DeltaXML technology provides the technology behind this, to allow changes to be marked up on demand.

Where does government data fit in?

The pandemic has made the Pandora’s box of government data transparent, open, accessible, and useable to netizens in every country. When change happens, it matters, and tools like those from DeltaXML are helping organisations around the world to be at the forefront of understanding change in structured content and data. There is responsibility, now more than ever, that data is not misrepresented or misunderstood, to ensure credibility and authority of data sources such as public institutions.


References

UK Government data source for daily data sets for infection rates, cases, deaths, and vaccinations – https://coronavirus.data.gov.uk/

Keep Reading

Introducing Character By Character Comparison

/
Find even the smallest differences in your documents with speed and precision with character by character comparison.

Everything Great About DeltaJSON

/
Accessible through an intuitive online GUI or REST API, DeltaJSON is the complete package for managing changing JSON data. Learn everything about makes DeltaJSON great.

Simplifying Your JSON Management Experience with DeltaJSON

/
DeltaJSON simplifies JSON data management with the introduction of an NPM package.

Cyber Resilience for SMEs: A Chat with DeltaXML’s Systems Administrator

Peter Anderson, IT System Administrator, relays the importance of cyber resilience for SMEs.

S1000D and Beyond: Navigating the Skies of Aviation Data with XML

/
This blog explores the significance of XML in aviation data management, highlighting standards like S1000D.

File Formats and ConversionQA Functionality

ConversionQA is a tool by DeltaXML ensuring the success of content conversion projects by comparing content from any two XML formats.

Introducing ConversionQA

ConversionQA is introduced as a solution to comparing content across different XML formats, addressing scenarios like content conversion and restructuring documents.

Making Tax Digital: Embracing XML Technology for HMRC Compliance

The Making Tax Digital (MTD) initiative by HMRC aims to digitise the UK tax system, but what does that mean for UK businesses?