The changing face of enterprise storage in government

A new breed of enterprise-class, cloud-based storage platforms is offering new hope for storage-weary administrators.

Storage has always been a bugbear for IT managers — even more so in large-scale government environments where heterogeneity rears its ugly head and criticalities such as data protection and governance complicate matters further. Yet while expertise in the creation and management of storage area networks (SANs) long ago became a necessary part of doing business, the increasing bulk of enterprise-class, cloud-based storage platforms is finally offering new options for storage-weary administrators.

Storage-in-the-cloud services comprise a growing portion of the infrastructure-as-a-service (IaaS) market, which Gartner has pegged at growing to nearly US$16.5 billion in 2015 — a 32.8% increase on 2014’s figures. This explosive growth rate is expected to continue at 29.1%, driven by growing demand from CIOs who, the research firm says, are increasingly turning to cloud-first options and embracing IaaS-based services.

Yet even in this climate of growth, providers are learning the hard way that success in the IaaS market requires more than just lots of drives running in a far-off data centre. Australian cloud-storage provider Ninefold, for one, debuted nearly five years ago as an early and enthusiastic proponent of the cloud-storage model but in November announced it would shutter its business at the end of January after concluding it had insufficient ability to invest “to make what we’ve built go to the next level”.

In retrospect, Ninefold’s error may have been that it tried too hard to commoditise cloud storage — renting out ‘instant horizontal and vertical scaling’ by the hour and offering easy integration via API. Or it may have just been a casualty of the surging globalisation of the cloud-storage market dominated by the likes of Google, Amazon Web Services and Microsoft — the three providers Gartner singled out as having most of the public-cloud IaaS workloads running in 2015.

Abstracting the enterprise storage

If recent market moves are any indication, Ninefold’s demise may also have been hastened by the explosion of cloud-storage providers that are increasingly looking to offer high-level value-add around the basic storage function.

Contending that storage as a service is not enough in its own right, such providers are wrapping the function of file storage inside layers of management and compliance that reposition them within the pantheon of IT solutions — and make them more immediately relevant for public and private sector players alike.

Vendors such as email-archiving stalwart Mimecast were early into this game, where it now butts heads against cloud email-archiving and compliance solutions such as Barracuda, MailStore, MessageLogic, Rackspace Email Archiving, Sonian Central Archive, Symantec Enterprise Vault, McAfee SaaS Email Archiving, Google Vault for Work and others.

The value-added approach has recently expanded from email archiving towards a more comprehensive IaaS offering that has proven fruitful for companies such as Simplivity and Nutanix, which have worked to abstract core infrastructure components and deliver them to working environments as increasingly self-managed ‘hyperconverged’ environments bundling storage with overlying layers of service management and other functionality.

That approach — which Nutanix calls ‘web-scale infrastructure’ — has proved particularly appealing for the Commonwealth Treasury, which recently worked with Canberra services provider Qirx to implement the Nutanix backend platform in support of a virtual desktop infrastructure (VDI) deployment incorporating as many as 850 fixed and mobile endpoints. (See our interview with Ricardo Alberto, Treasury’s CTO.)

The Credit and Investment Ombudsman (CIO) has seen similar improvements after implementing the SimpliVity OmniCube platform into its primary data centre and disaster recovery sites. The environment — which wraps storage and nearly a dozen other core functions below a VMware hypervisor — has simplified the functioning of the agency’s compute environment and relieved management pressures that were growing as the agency struggled to keep up with a growing volume of data.

“We just didn’t have the human capital to manage VMware running on a complex SAN with traditional servers,” IT manager Matt Grech explained. “We were trying to figure out how to get the most resource-efficient solution from a capex and opex perspective.”

The SimpliVity environment isn’t cloud-hosted but, in federating and effectively virtualising the various functions of the enterprise infrastructure, it is logically similar to the emerging set-and-forget, IaaS-based cloud-storage models.

The move has helped the CIO shift towards a continuous backup regime, with better off-site data protection and disaster recovery. Storage consolidation has led to a 295-to-1 improvement in data efficiency, threefold reduction in rack space and fourfold performance gains for report generation — and shifted the organisation to a virtualised core infrastructure that largely looks after itself.

This improved backup and storage management approach recently paid off after a core accounting application was corrupted and, rather than running an hours-long manual recovery process as in the past, IT staff were able to restore the data within minutes by using a backup taken just 15 minutes earlier.

Formalising the cloud conceit

The increased operational value provided by largely self-managing infrastructure may already be delivering wins — but it is also setting the stage for the transition to increasing use of cloud-based storage services not just as file lockers, but as seamless, compliance-driven extensions of government agencies’ infrastructures.

Enterprise-storage giants EMC and VMware recently pushed into this space to join the likes of Commvault, which recently updated its core Commvault Data Platform (CDP) portfolio, an open-standards platform that breaks down the barriers between on-premise, hybrid, private-cloud and public-cloud environments like Amazon S3 Cloud Storage and that company’s Glacier large-file archiving service.

Simply shuttling data to and from S3 and Glacier isn’t enough, however: today’s unified data management approach includes data-aware governance controls, security, backup and replication capabilities, extensive searching and a host of other solutions that deal with data in the cloud as easily as they deal with data on premises.

Commvault is aiming high with CDP, which it is promoting to partners and ISVs as a core element of a broader storage and data management infrastructure. Conventional backup techniques, such as snapshotting and disaster recovery, are being paired with active data management and wrapped into a single management framework designed to simplify the task of keeping on top of the environment in real time.

Easier availability of such services is driving expansion in offerings such as BCT Australia’s compliance-driven backup-as-a-service offering, which offers multiple site redundancy for storage of not only email but files and folders, database files and server configuration information.

This sort of approach is being writ large within government agencies as states such as NSW push hard to consolidate core services and deliver data analytics and other key capabilities. This push towards consolidation complements the evolving effort to bury storage in layers of management and integration — such as the effort undertaken by consultancy Secure Logic to work on the NSW Government’s recent GovDC data centre consolidation.

Secure Logic began working with the Department of Finance, Services and Innovation several years ago to simplify and consolidate its infrastructure, combining a broad range of capabilities into two core data centres. To replace a morass of stand-alone solutions, the focus of the project was to build storage and other services into an easily manageable environment that would allow secure access to a range of service provider offerings and scale to support the needs of the various government departments relying on the new environment.

“What we’ve done is to take a lot of complex solutions and turn them into a very simple, straightforward way to deliver,” explained CEO Santosh Devaraj. “We took the department’s vision and created a technology ecosystem blueprint where agencies can consume their information or services from any supplier. It doesn’t matter whether it’s a small organisation or a large enterprise: they have an equal chance of offering the solution to any other agency or service provider offering the solution as well.”

By presenting itself as a cloud environment to state government agencies, this environment can offer encrypted, securely managed data storage on an IaaS basis that is both physically removed from the requesting agency and tightly manageable by that agency as part of its overall storage environment.

The role of cloud in empowering this ecosystem is set to expand in 2016, Devaraj says, with new service providers and security capabilities coming online as well as cloud-connection tools and aggregators for cloud services. In this way, the environment will continue to empower IaaS providers to improve their own cloud-storage environments while allowing users to seamlessly interact with those environments on an as-a-service basis.

“The cloud has become a big thing now,” said Devaraj. “As agencies move into GovDC, there is a whole lot of migration effort required. It truly comes to the point where they analyse what’s there, the metrics of the service backbone and the benefits of consuming a range of services from the market provider. By leveraging the consolidated model, they can drive the cost down to make [cloud] a very easy point of comparison for them.”

Image courtesy Andreas Wienemann under CC