From open public data to open 'any' data

By 2018, a third of digital government projects will treat any data as open data. But how do we get there from here?

To date, the value to society from the one million (and counting) government open datasets has largely been achieved through efficiencies realised in the form of savings or economic surplus, such as reducing commute times, rather than by generating sustained revenue. The exceptions are the use of weather and climate data to improve forecasting models and the use of GIS data to create new services for business and citizens.

Growth in the established commercial markets or entrepreneurial start-ups that profit from government open data will continue as more diverse and rich datasets are opened for public consumption. New markets will develop as open data standards in areas such as personal health management or residential or business energy consumption become widely adopted.

However, the benefits to government to sponsor and maintain an open data program have been indirect and difficult to measure. To obtain quantifiable business value from its data assets, government must identify and pursue open data use cases that produce additional revenue, improve business operations or achieve better program outcomes.

For example, increased compliance may result when tax and revenue data is used internally by a licensing and permitting agency to scan transactions involving properties of individuals or companies owing taxes. At the same time, Gartner advises program managers to track and publish the costs associated with each open dataset. Posting the cost of producing open data is one way to determine its value to society and is a reminder that ‘open data is not free’.

Through all stages of the information life cycle, government information management practices must evolve to allow any service or application to use or re-use data within the parameters of its access rights. The seamless exchange of government data across agencies, jurisdictions and industries requires the end-to-end digitalisation of business processes and data accessible through web APIs. What will emerge are transformed business processes or radically new service delivery models enabled by the flow of open data among government agencies, citizens, businesses and things.

The role of legacy modernisation

Many government organisations are struggling with modernising their legacy mission-critical systems by using a combination of commercial off-the-shelf (COTS), open-source, re-usable and externally sourced (including cloud-based) solutions. At the same time, these organisations are involved, to different extents and degrees of maturity, in a number of open government initiatives that often revolve around making public data more easily accessible through web APIs.

These two activity streams may be much more closely related than many people think. Better integration and data exchange between mission-specific applications and more accurate data analysis capabilities to improve performance are top of mind for many government CIOs, and yet they rarely connect the dots between those problems and how open data may offer a solution. Gartner recommends that government CIOs identify low- to moderate-risk legacy modernisation initiatives in which an open-data-centric approach can be used.

Many tend to equate open data with public data, given its original definition. However, data can be defined as open when it is machine-readable and is accessible through an API. This can apply to potentially any data that needs to be processed: whether it is public, discoverable through Freedom of Information Act requests or restricted (for example, covered by privacy laws).

The value of public open data resides in increasing transparency, allowing internal and external parties to figure out new ways to use data that can deliver efficiency or even contribute to economic growth.

Efforts are already underway in several jurisdictions, such as the UK, to move from open public data to linked public data. The use of linked data supports easier sharing and integration of data across enterprise boundaries. Gartner defines linked data as a web-orientated set of technologies and methods that simplify the publishing, discovery, interoperability and re-use of data for the purpose of generating information-sharing network effects. ‘Linked data’ is a data management and mathematical principle, which holds that any time two data points are used together it creates a weighted link. As more valid use cases for the link emerge, the link grows in weight.

Also, relentlessly opening data allows the uncovering of so-called ‘dark data’ — that is, information assets that organisations collect, process and store in the course of regular business activity but generally fail to use for other purposes.

Towards data-centric government

The next, more disruptive step is to consider usually restricted business-specific and personal data as open data. This data is not meant for public consumption, and is fraught with privacy and sensitivity issues, yet it can be modelled as open data to facilitate the development of more granular and agile context-sensitive applications, as well as a more coherent data exchange and analysis across agency boundaries and beyond.

There is only anecdotal evidence that this is happening, but this is a key trend for governments that want to become smarter — that is, more affordable and sustainable.

This approach gives rise to what we call data-centric government. The focus is no longer on applications. Data is now the key asset, around which applications are built.

Data centricity in government has several advantages. It supports:

• Better interoperability and joining-up. Rather than being forced to extract data from applications to achieve integration across organisational boundaries, data is described and accessible through a web API by all prospective user applications according to specific access rights.
• Innovative application development and procurement. The same approach that is used to develop mobile or web applications based on open public data (such as app contests, hackathons or datapaloozas) can be used to build applications that access non-public data. This would favour more agile development and support greater employee centricity, as employees would be able to develop and/or compose applications to access data in more effective and convenient ways.
• The evolution towards ‘citizen data vaults’, giving citizens better ability to control access to their own data and share it across agencies or with the private sector if they wish.

Data centricity also carries a number of challenges:

• By helping to break silos, data centricity threatens the status quo and thus may encounter all sorts of resistance, usually expressed in terms of security and privacy implications. Open data does not pose a threat to either per se as access to data can be controlled in an even more granular and auditable way. However, the proprietary attitudes of programs or agencies that generate data can also have a chilling effect on open data and data-sharing initiatives. Data ‘ownership’ issues are often as big a barrier as security or privacy, so open data governance that operates according to the principal ‘government data is a public good’ is key.
• Open data and public data are often synonymous or used interchangeably, and this leads to possible confusion. Senior leaders do understand the importance of open public data and want to be seen as progressive and transparent, while they are unlikely to see any political capital in focusing open data on more internal issues, such as productivity and integration.
• Required skills for data architecture are in short supply, and open data expertise is mostly available in the Web 2.0 and open government circles, where people are mostly concerned with transparency and cool apps.

The confluence of open data and legacy modernisation in government is creating the conditions for a new way for CIOs to structure public services and applications that is centred on data rather than processes. The growing public use of government open data has demonstrated its potential for economic development, increased citizen engagement and the formation of entrepreneurial start-ups. The end-to-end digitalisation of business operations, and the digital data generated by those operations, will create new service delivery channels supported by public-private partnerships. Innovation within government and across other sectors of the economy occurs when data is easily collected, published and re-used, no matter where it originates — be it from people, digital business operations, or things such as sensors or devices.

Remember that open data is not synonymous with public data. An increasing number of organisations are leveraging open data for internal transparency and data integration purposes. Although the political capital of open data resides in its external consumption, government organisations can greatly benefit from it internally.