Postcodes no longer spatial gold standard: Brobst

Governments hoping to make better use of spatial data need to revise their thinking around the level of data granularity that is acceptable for making actionable policy decisions, a leading US government advisor and data-warehousing executive has warned. 

“Our government customers are moving towards a model where location is no longer a postal code,” Stephen Brobst, chief technology officer with Teradata Corporation and a member of US president Barack Obama’s President’s Innovation and Technology Advisory Committee (PITAC) told GTR.

“Postal codes are optimised for delivering mail and not much else; they don’t make much sense for decision making. It’s now all about latitude and longitude, and you collect a huge amount of data over time.”

Tracing spatial information to extremely high resolution may improve the accuracy and relevance of decision-making, but it also creates a massive impost in effort and data-processing demand that will send many government bodies outsourcing management and access to many data sets to cloud-storage providers. This represents a major shift in philosophical approach from earlier days, where geographical information systems (GIS) were built as massive silos of data that were only accessible by specialised staff within government departments.

Those days, Brobst said, are gone. “Siloed GIS systems are a thing of the past,” he explained. “Today it’s really much more about integrating the geospatial data with other, more traditional data. You shouldn’t have to go off to a separate system to do that kind of analysis.” Challenges in data management persist, however – particularly in terms of the sheer number of information types being managed. A growing reliance on sensor data, for example, as well as unstructured data from less-formal data sets, means government bodies still need to dedicate significant energy to data stewardship. “The bigness of the data is fundamentally less interesting than the diversity of the data,” Brobst said. “We’re going to go from analysing transactions, to analysing interactions – the new kinds of data below the transactions. Often this data comes in forms that are not aligned to traditional database structures – so you need new ways of storing and analysing that data.” As the early success of government-sponsored hacking competitions have shown, government bodies aren’t on their own when it comes to finding those new ways. After all, one benefit of broader access to traditionally tightly-held spatial data sets is that the public and private-sector interests can contribute to the effort by developing innovative uses for high-resolution data.

“The idea is to let the marketplace figure out what the uses for the data are,” Brobst explained. “They’re making a bet that there are enough creative people out there to build the things that we would never have conceived of – or the things that politically government bodies aren’t allowed to build.”

Yet there are lingering issues in supporting this kind of open data environment, scalability being one of the major ones. Although the data may be of little sensitivity and therefore fine for public consumption, privacy protections, performance and other issues will still force governments to tread carefully – and drive many to ensconce their data in private rather than public clouds.

“Government organisations are large enough that they’ve got the economies of scale so a private cloud infrastructure is very cost- effective and quite viable,” Brobst explained. “It’s probably more cost-effective to have a private cloud; remember that public clouds are not charities, and there’s typically a 30% margin built in. The question is: can you operate your data centres at 30% of the best of the best?” – David Braue 

This story originally ran in the June-July 2012 issue of Government Technology Review.