Big data doesn't mean all data

Tight budget pressures mean many government agencies are embracing big data more quickly and effectively than their private-sector counterparts, a big-data analytics specialist has found after working with a number of agencies to plan and execute their big-data strategies.

“The concept of big data and analytics has yet to be properly defined, certainly in the Australian market,” says Dr Theo Gazos, a former KPMG and PwC director who is now director and co-founder of analytics consultancy Predictive Analytics Group (PAG).

Gazos says his work with government bodies suggests that they are “a bit ahead of most private organisations” in this process because they are driven by tight budgets and business cases face strong competition between departments when angling for precious IT budget.

In many cases, this means working against the assumption that big data necessarily involves the collection of every single piece of data; rather, Gazos says, the focus of big-data planning should first look at what data is already being collected and how it can be more efficiently used.

“Most of the conversations about big data are about the quantum of information,” he explains, “but we tend to tell organisations that, before they run off and spend tens of millions on technology designed to capture all this information, that they think about analysing the information they currently have and collect.”

“Make sure the analytics you're providing are robust and unique,” he continues, “and that they produce reliable forecasts. Then think about the information that's missing. Agencies should target their future investments to collect that information, instead of just assuming that everything out there is required and is going to be useful.”

One current engagement has seen PAG working with Victoria's Emergency Services Telecommunications Authority (ESTA), which has been using big-data techniques to model its contact-centre activities.

While the engagement involved establishing a baseline of ESTA's activities, it became clear that other data – specifically, Australian Bureau of Statistics figures on population growth – would add additional value by allowing models of current demand to be meaningfully extrapolated into the future.

“They want to be able to produce forecasting for the future and understand how call-taking volumes, for example, will change as the population changes,” Gazos explains.

“In developing those mathematical models there's a lot of work that goes into identifying, in this case, the relationship between 000 calls, crime rates, population trends, and so on. They've got potentially thousands of calls coming in per day, and this gives them an opportunity to say 'what if?', and to simulate the outcomes of various things occurring simultaneously.”

The promise or demonstrable reality of such value is helping legitimise targeted analytics approaches as business leaders work to build viable business cases to justify big-data projects.

“They recognise the value of providing a robust business case to treasuries and central purchasing agencies,” Gazos says.  “The value is in having a model that lets you look at the impact of many changes simultaneously. It's creating an evidence base of information, and providing a more convincing argument when you're pitching for funds.”

Each big-data analytics model will be different, Gazos says, since it's based on the unique characteristics of each organisation adopting the technology. Extensive customisation of analytical models may require specialist skills and go against many organisations' predilection for low-risk, off-the-shelf solutions, but it also provides a competitive advantage that is tailored to the business and cannot be easily replicated by others.

Ultimately, big data is about unearthing new insights in the data that organisations have, or collect into the future.

“It's only through careful analysis of data that organisations will be able to answer some of those questions,” Gazos says. “You need to make sure the mathematics you're using to generate your forecasts are solid and robust. That doesn't mean they have to be complex; they just have to be correct.” – David Braue