Governments must get creative with data reuse: EMC

The growth in data volumes being collected and analysed will drive significant changes in the behaviour of government departments, EMC marketing chief technology officer Clive Gold has warned — but without prudent cloud investments, many will struggle to deliver.

“Big data has already started to change your life,” he told attendees at last year's Cloud Computing Forum 2012. “The entire existence of humans is now digital — but we have no clue about how to handle so much stuff. If you’re in government, it’s going to change the way you govern."

The data is out there – but it's up to governments to help citizens make the most of it, EMC's Clive Gold argues.

Everything from consumer services like Facebook and Twitter, to massive data-gathering projects like the proposed Square Kilometre Array (SKA), is adding to the volume of data available for analysis, Gold said — and government agencies have the opportunity to take a lead role in figuring out new things to do with it.

An avid biker, Gold cited the example of Strava, an iPhone app that tracks bike rides, then aggregates and compares them with rides recorded by others in the same area. A government road authority could, he said, offer great value by providing accident data to a vendor like Strava so that users could easily see and avoid areas where the highest rates of accidents occur.

Similarly, meteorological authorities could add in information about weather conditions at the time, over time establishing a correlation between the accidents and inclement conditions.

“We’ve got to think about how we bring this data together,” Gold said. “There’s real power and value when we start linking together things that we never thought to link together before. By bringing together people, places and events, you can start better figuring out what’s happening.”

Getting to that point, however, will require a change in the way government bodies approach data analysis: while they may be good at collecting data, unstructured data defies the strict controls that departments have typically been placed on data through the use of data warehouses and relational databases.

"The traditional data warehouse is built on relational technologies that are not designed for” dealing with unstructured data, Gold explained. “It was a disjointed process. Imagine if Facebook worked like that — but it is a collaborative environment and new technologies are based on the idea that the environment should enable you to pull out something you want.”

Cloud providers, many of whom are pushing notions of cloud-based big-data analysis, have taken a lead in helping client organisations make sense of this data.

“Now, all the technologies are there for combining structured and unstructured data,” Gold concluded. “But you have to redefine the questions in order to get the data in such a way that you can try and answer the questions. Cloud providers haven’t just taught us how IT can be used differently; they’re actually teaching us how to use masses of data, unstructured and structured, to build on the value that IT can bring to the organisation.” – David Braue

This feature originally appeared in the February/March 2012 issue of Government Technology Review. Clive Gold was presenting at the Cloud Computing Forum 2012 conference in Canberra.

The Cloud Computing Forum 2013 will be held at Rydges Lakeside, Canberra on 20 and 21 February 2013. Registrations are currently open.