Open data program to help reduce natural disaster risk

Since 2010, natural disasters have impacted thousands of citizens and cost New Zealand close to $27 billion in insured losses.

Freely and easily accessible data on earthquakes, tsunamis and other natural hazards can help accelerate global research. It also enables governments and communities to better understand and prepare for natural hazards, reduce risks to prevent disasters when events occur, and even forecast future events. GNS Science Te Pū Ao (GNS Science) is New Zealand’s leading geoscience and data provider. Its GeoNet program uses the Amazon Web Services (AWS) Open Data Sponsorship Program to achieve this.

This means that GeoNet’s entire archive is available to download for the first time. The program covers the cost of storage for publicly available cloud datasets to make them accessible for analysis on AWS and encourages the development of communities that can benefit from shared data.

The GeoNet program is a geohazards monitoring service that produces data that captures earthquakes, volcanic activity, large landslides, and tsunamis. GeoNet is run by GNS Science with primary funding from Toka Tū Ake EQC (formerly the Earthquake Commission). GeoNet also collects data between geohazard events such as ambient noise tomography, which records seismic noise waves from Earth’s vibrations, and allows imaging of the planet’s subsurface. These images provide insights into the tectonic structures underpinning New Zealand’s geological hazards, and are highly valued by researchers.

Making data freely available to researchers has been an underpinning philosophy of the GeoNet program since its inception in 2001. Now, its datasets are faster to download, especially for large dataset requests, and more accessible to researchers, learning institutions, and the general public globally via the Registry of Open Data on AWS, which exists to help people discover and share data.

Recording data on the ground to safeguard people

By using the AWS Open Data Sponsorship Program, GeoNet is able to eliminate unexpected costs associated with users downloading large datasets, while providing end-users with free and open access to its entire archive. This data can be used to leverage machine learning services to improve natural disaster monitoring that benefits the broader global community.

For example, GeoNet data is used as an input to hazard models that inform New Zealand engineering guidance, which means structural engineers can better place critical infrastructure like power plants based on the probability of an earthquake, landslide or tsunami. The insurance industry uses hazard and risk models, with input from GeoNet data, to better calculate risks of disaster-prone areas so that actuaries can better estimate the financial impact of uncertainty for insurance policies and help clients minimise risk.

“Machine Learning is opening up new research avenues all over the world. Enabling access to our large and fundamental geohazard datasets and helping users bring their compute to the cloud is a key step forward because of quicker and easier access to information,” said Dr Jonathan Hanson, GeoNet Data Manager, GNS Science.

“Being part of the AWS Open Data Sponsorship Program is a great opportunity for us to step boldly forward, supporting research that could have wide-ranging dividends, like training algorithms to detect currently unknown trends that could potentially predict future events.”

Dr Calum Chamberlain, a geophysics lecturer from Te Herenga Waka — Victoria University of Wellington says the move will greatly benefit researchers.

“GeoNet’s high-quality seismic data is at the heart of research projects around the world. By making their data open-access on one of the most broadly adopted cloud computing platforms, AWS, GeoNet will open the door to researchers to apply cutting-edge methods to long-duration seismic datasets to understand earthquake processes for the benefit of New Zealand. Personally, having cloud access to GeoNet’s seismic data has helped me work on problems of long-term earthquake interaction and aftershock forecasting,” Chamberlain said.

Huge demand for data and knowledge

Since its inception, GeoNet has accumulated over 100 TB of data collected by its seismic sensors, coastal sea level gauges, geodetic instruments, acoustic sensors, tsunami sensors, cameras and environmental sensors to help us understand our planet. This is the equivalent to 4.5 years of HD video, and the data is growing by 13 TB, or as much as 6500 hours of HD video, annually.

More than 100 TB of GeoNet’s data has been accessed by end-users around the world in the last 12 months, illustrating the huge demand for knowledge. Information about the GeoNet public dataset can be accessed by researchers, students and education institutions to discover and share datasets to enable global collaboration and accelerate research.

Prior to joining the AWS Open Data Sponsorship Program, GeoNet manually handled requests for large amounts of data from researchers, filtering and exporting datasets to an external access point which researchers collected. This process was slow and labour-intensive, previously taking a number of weeks, compared to instant access that users now enjoy. It was not previously possible for users to download the entire GeoNet seismic archive. Moving to the AWS Open Data Sponsorship Program allowed GeoNet to offer instant access to large datasets and the entire archive at no cost.

Toka Tū Ake EQC Manager Research Dr Natalie Balfour says having open access to hazards data and information and getting the data to the people who can use it will have a global impact.

“New Zealand’s data is of great value for the science community, universities, engineers and so many others worldwide. We’re thrilled that AWS supports our commitment to sharing data and sponsoring GeoNet in this venture. To have this data freely available on AWS will drive research globally, especially in areas like machine learning, and help us to better prepare for natural disasters,” Balfour said.

The GeoNet program is a collaboration between GNS Science Te Pū Ao, Toka Tū Ake EQC, Toitū Te Whenua Land Information New Zealand (LINZ), Hīkina Whakatutuki Ministry of Building, Innovation and Employment (MBIE) and Te Rākau Whakamarumaru National Emergency Management Agency (NEMA).

Image: Papatea Fault rupture — Kaikōura Earthquake, November 2016. ©Julian Thomson, GNS Science.