Sparx Systems advanced approach to data management

From cuneiform script on Sumerian clay tablets to e-books, and from the first codification of the constellations in Ptolemy’s Almagest to the Square Kilometre Array radio telescope, history tells us the information and business models of today will be replaced by new and different models tomorrow. We are at the start of a global industry change that is being driven by legislation, standards and technology, and fuelled by information (big data).

This change contains the promise of unprecedented potential, including the ability to create more accountable, efficient, responsive, and effective governments and businesses, and to spur economic growth. The factors currently driving this change (social media, web and software logs, cameras, information-sensing mobile devices, aerial sensory technologies, genomics, and medical records to name a few) form a confluence referred to by Gartner (the world’s leading information technology research and advisory company) as the Nexus of Forces.

It has been estimated the world produced 14.7 exabytes of new information in 2008, nearly triple the volume of information in 2003. According to IDC, the ‘digital universe’ – a measure of all the digital data created, replicated and consumed in a single year – is expected to grow by 2020 to 40,000 exabytes, or 40 trillion gigabytes.  We are in the era of big data and the data management challenge is daunting. The frantic pace of data growth can appear overwhelming.

The LSST
Without an efficient information management environment in which to store, search, share, visualise and analyse large data sets, big data cannot be effectively used. Gartner notes that, while big data will make organisations smarter, enterprise architects will play a major role in ensuring their organisations maximise associated business opportunities, while the disruptive effect of big data can be tamed with enterprise architecture.

The Large Synoptic Survey Telescope (LSST) project, which is moving towards the start of federal construction in 2014, is expected to acquire 150 terabytes of data every five days. Astronomers will use the new synoptic telescope to view and photograph a quarter of visible space every night, starting sometime early in the next decade.

Data management is one of the most challenging aspects of the LSST, as data must be processed and stored each night, producing the largest non-proprietary data set in the world. Sparx Systems Enterprise Architect was chosen as the modelling environment because the project needed a multi-user, full-featured UML tool with traceability support from requirements to generated code.

Official policy
President Obama signed the Open Data Policy on May 9 2013. It describes information as a “valuable national resource and a strategic asset to the Federal Government, its partners and the public”. Open data can generate new businesses and stimulate growth. Many of the advances in mobile networks are the result of government making spectrum available to industry. Similarly, advances in location-based applications can be expected from government making data available: for example, the European Commission INSPIRE programme. However, open data is interoperable data that enables distribution. The more it is distributed, the more it is used. The more it is used, the greater will be the return on investment. Gartner has said that open data will be far more consequential than big data for increasing revenue and business value in today’s highly competitive environments.

However, Open Government Data (OGD) practices are still in their infancy, according to Sir Tim Berners-Lee, co-founder of the Open Data Institute. In a report called the Open Data Barometer, published in October 2013, from a survey of 77 countries, the UK led the top performers Sweden, New Zealand, the US, Norway and Denmark, while Nigeria, Mali and Saudi Arabia scored the lowest. The report noted that, while “OGD policies have spread fast, the availability of truly open data remains low, with less than seven percent of the dataset surveyed in the Barometer published both in bulk machine-readable forms, and under open licenses”.

Meanwhile, the super abundance of data with which to create profit and competitive advantage comes at a risk: data providers, both public and private, must now navigate tighter than ever privacy laws, breaches of which carry hefty fines. Changes to Australia’s Privacy Act, passed last year, require companies that collect and store personal data to do so openly and transparently. This necessitates knowing where sensitive data resides and masking it effectively. Gartner notes this is where enterprise architects can help as navigators of strategic change, charting the right course for big data across the most critical dimensions of the organisation: business, culture, talent and technology.

Metadata structures
On the principle of data quality, the goal of the Open Data Charter signed at the 2013 G8 Summit in Northern Ireland is to ensure the releases of “high-quality open data that are timely, comprehensive and accurate. To the extent possible, data will be in their original, unmodified form and at the finest level of granularity available”. Granularity impacts creation and capture. However, the greatest impact of granularity is in maintenance.

High levels of granularity support more structured information and allow more technical manipulation. Metadata with levels of low granularity can be created for much less cost but provide less detailed information. As the metadata structures become out-dated, access to the referred data will become much harder to manage.

Ensuring the longevity of information as a valuable asset will, in many cases, require investment in new data capabilities. A metadata repository enables collection, storage, maintenance and dissemination of metadata information while managing the structure and transformation of the metadata, from one structure to another, as new systems get developed. Enterprise Architect is a powerful and inexpensive repository tool that can be used with a variety of databases.

Enterprise architecture
The growth in information volume, velocity, variety and complexity, and the increased importance of information to the business makes the discipline of information management radically different from the past. Old techniques are no longer adequate: a dramatically different approach is needed. Increased conformance to open standards opens organisations to innovation.

Quantum leaps and leading-edge technology deployments are made possible by interoperable information sharing and enterprise architecture, which create increased flexibility and control over business operations. The advantages of having a ‘standards-driven’ enterprise architecture include improved decision-making, adaptability to changing demands or market conditions, elimination of inefficient and redundant processes, optimisation of the use of organisational assets, and minimisation of employee turnover.

Sparx Systems Enterprise Architect is a team-based modelling tool that is used for bi-directional communications across a project and is geared towards the capturing, definition, visualisation, management and sharing of big data and their related schemas. It connects business experts and subject area experts from the architect and business analyst, to the people responsible for technical implementation. The idea is to organise the elements of a project into a continuous flow of artefacts, from the business requirements to the eventual solution. For the fourth consecutive year, Sparx Systems has been recognised by Gartner in its Magic Quadrant for Enterprise Architecture Tools. This is largely because of Sparx’s flagship Enterprise Architect modelling software, and was placed in the ‘Niche’ quadrant by Gartner analysts.