ABS tries not to become a virtualisation statistic

Like many government organisations, the Australian Bureau of Statistics (ABS) has invested heavily in virtualisation platforms to consolidate its servers, speed provisioning times and take advantage of a far more flexible infrastructure. Unlike many organisations, however, the ABS recognised early on that doing so would require a significant change in its security posture.

This realisation came as the ABS undertook a massive migration effort that saw it virtualise around 99 percent of its servers, shifting applications on to what became around 1500 virtual machines (VMs) running within a VMware ESX Server environment.

Nearly every aspect of the ABS’ administrative systems was touched by the move, which saw email, calendaring, workflow systems, databases and other applications running on high-utilisation virtualised servers. If a single machine fails, a replacement can be quickly commissioned on a different server to minimise functional downtime.

While it provides very real operational benefits, the virtualisation shift fundamentally changes an organisatoin’s security profile: not only do individual VMs have to be secured, but the hypervisors running them also need to be addressed because their interrelationship with the servers offers a range of new attack vectors for malware. This made it untenable to simply port existing applications onto virtual servers, then expect the same security protections to offer adequate protection.

“In traditional security you’ve got a big firewall on the one end, and everything in the middle is fairly open,” explains Fred Donnelly, team leader of security operations with the ABS. “Virtualisation has given us a lot of flexibility, but as we did that we saw a need to protect those assets on the internal network. “

That requirement was further driven by the growing usage of personal devices on the organisational network — an issue that was almost non-existent even five years ago. These days, however, the onslaught of personal smartphones and tablets further contributes to the complexity of the environment by potentially introducing new threats inside the firewall; with 3500 employees at the ABS, the extra risk of exposure is significant.

These demands created further imperative for the protection of networked hypervisors using security capabilities that a firewall alone couldn’t hope to provide. After evaluating a range of available options, the ABS elected to implement Trend Micro Deep Security, a multi-modal platform that combines intrusion detection security (IDS), intrusion protection security (IPS), zero-day threat protection via deep packet inspection, log file monitoring, and host integrity monitoring.

Deep Security ties these capabilities into the virtualisation platform by utilising a range of VMware APIs to hook into the hypervisor and set up ‘virtual gateways’ protecting access to the virtualised servers. The platform scans files at the hypervisor layer and performs continuous integrity checking to ensure the foundations of the virtualisation platform aren’t being eaten away by network malware. Importantly, it functions regardless of whether the VMs are active or not, and supports agents running on multiple operating systems.

“It allows us to control the stuff at the network layer, and gives a lot more visibility,” says Donnelly. “Having it all at the hypervisor level means that if something goes wrong with the machine, or you’re running up a new machine, it’s got that basic level of protection. Having that level of protection closer to the data makes that much more sense.”

Tweaking for security. Although the default installation is relatively easy to get up and running, it was only the beginning for the ABS team, which explored other ways to minimise its exposure through the use of virtual machine configuration.

One option, for example, is to configure new virtual machines so that they cannot operate until they’ve downloaded and installed the latest security patches. Use of agents also proved valuable, allowing the team to manage the VMs and their risk profiles using a range of security controls with features that aren’t available through the basic API-driven hypervisor hooks.

“The more effort you put into tying down those VMs to minimum requirements,” Donnelly says, “the more benefit you’ll get out of making sure the machines can only do on the network exactly what you expect them to.”

Development of a strategy for ongoing monitoring of the security profile has proved crucial, with a steady stream of questionable events being intercepted on the network and measures of baseline activity continually being revised based on changing network usage patterns.

Interpretative skills are essential in this regard: without skilled IT staff that know how to use the tools at hand, raw firewall dumps and other large reporting by-products become unwieldy and difficult to deal with.

“You still require people with the right skills to understand what they’re seeing, but the tools help gather intelligence and clear out some of the background noise,” says Donnelly. “You’re always relying on one vendor’s view of what’s important and what’s not, but it is a balancing act around what’s safe to consolidate and what’s not.”

One issue that has required particular attention for the ABS is backup, which has become a different entity entirely with the introduction of virtualisation. Although hypervisors provide some measure of backup staggering and management, the addition of regular security scans has increased the need to make sure multiple VMs don’t choke the server by running disk-intensive processes at the same time as backups. To resolve this potential issue, VMs are scheduled in patterns rather than being set individually to specific times; this lets the platform best manage backups and scans.

With continuous tweaking the order of the day, the security team has turned its sights to future improvements such as the introduction of two-factor authentication to all users, and improvements to remote access and mobility that will both modernise and further secure the organisation’s infrastructure.

With such a high level of virtualisation within the ABS, Donnelly says the key to successfully making the jump is to take a staggered approach rather than just treating the technology as a set-and-forget concern. Start small, make small changes, and watch their effect on core systems — and those they are interconnected with — before changing anything else.

“Any new layer of IT protection has a cost as far as CPU and disk, so you can’t just roll out the system everywhere and hope it works,” he advises.

“There’s no point in putting in any sort of security system unless you’ve got the resources to actually manage it, and make sure it’s kept in line with your business. And ultimately, it’s about being proactive in protecting our assets. Having that protection at the network layer across our whole environment means that if problems occur, we’ll know about it sooner and be able to rectify the situation.” – David Braue